Your story sounds EXACTLY! like mine! Same sypmtoms and all.
Not to make you sad, but we never got a diagnosis. Spent thousands at the vet, including an internal specialist, and never ever got results. The only thing we did not do was a biopsy, as I did not feel it justified to put my dog under a surgical procedure only to find a potential diagnosis that you would treat the same way as if you assume there is something. Plus, there were no guarantees it would find anything, it was very expensive, and it was right before a holiday after which he seemed to be getting better again.
Given that, it is up and down with him, there are times he is great, tolerates his food awesome, then times again where he is on the same food with bouts of diarrhea, and it seems that once a year he goes through a crazy weight-loss time.
We did find giardia this winter, which is difficult to diagnose and to get rid off, and part of me suspects that this is what he had all along and may still have. Although no one expected it as at that time point he didn't have diarrhea for 3 months, even the vet was astonished at that. There is no way to really prove that they are gone, but he has put weight back on and looks healthier again. Same as in 2014, when we just treated him for giardia not actually diagnosing them but not knowing what else to do as nothing else helped. So my suspicion is that he is sensitive to them and just gets them again and again. But I never will know that for sure! So maybe ask your vet to treat for giardia even without a diagnosis (which a good vet should recommend anyway given his history because giardia is almost impossible to actually diagnose, don't let a vet tell you any different!)
My boy is otherwise perfectly healthy, happy, active, etc. just re-curring digestions/GI issues, which sounds like your boy. So hopefully it helps you knowing that you are not alone with misterious GI issues :-D for starters!
In terms of food: I tried various kibbles, wet food and raw, including vet diets (Hills ID, which I got rid of asap, but wanted to follow the vets procedure at least for a little while so they couldn't keep saying it's the food..). He is currently on Hermann's duck and sweet potato and I add some 100% turkey meat, cooked eggs, salmon/tuna tins, cooked purreed veggies, etc. to spice it up. He was on Bosch kibble before that and before that on the ID. He did well on Bosch for a while but then got diarrhea again, turns out the food was contaminated with corn and who knows what, so I went away from the company and won't recommend them again. While I wish Hermanns was more open about their recipe, I appreciate their customer service.
I also worried about switiching too often and it always says to try for weeks, but to be honest my dog is different. If he does not tolerate something I will know after 1 or 2 times feeding it and it makes no sense to feed him longer with it. It is better to switch often for him to find something he tolerates. If it is in between I usually do try for longer, but until I found something that worked I switched him often and literally when he tolerated something he was better straight away. But each dog is different with that.
At one point I did a bioresonance energy testing. People do not necessarily believe in it, and many vets don't, but my friend had success determining her dog's allergies with it and it was only 60Euro, which is a lot less than anything I spent at the vet. I received a thorough list of his food intolerances, and I can say that it really helped me. It narrowed down the search and avoided an exclusion diet, which I was worried about doing given his loss of weight at that time. Whether 100% true or not, I give him food of the "no" list and he gets diarrhea, if on the "yes" list he is usually fine (depending on what it is it also depends on quantity of course). It is changing again a little with him being more stable now I think, but it gives me guidelines which is really helpful and I can try little things again as he is being stable.
Just be careful when you look for someone who offers it. There are no regulations, so anyone can offer, even if they don't know what they are doing. I did it in Germany, so not sure how it is over here.
Anyway, sorry, long response. I just know what you're going through, as it is the same for me! Glad to know I am not alone, and hope that some of this helps you and your dog too!