My older brother has been training for 5 years when he was in high school and it seems he kept his physique since. Even after 15 year of not training he still looks very athletic. I noticed this on some other folks who trained Judo. Is this something because of the sport itself or is this just a coincidence?
Judo is one of the most demanding sports out there believe it or not.
It's right next to basketball and soccer. You need to be in the great shape. Now, if you train Judo for 3 or more years it is fine to say that you are in great shape because of it.
There are many benefits of judo on human body; strength, speed, posture, gymnastic ability, endurance....list goes on.
Most of the judo trainees start at relatively low age, when they are still in development. And I think this is the answer you your question of effects of Judo on physical appearance later in life.
Because these kids start to work out their muscles at the same time when the develop it is more likely to stay that way during later life. They formed their muscles the right when during crucial time.