Came here to say something similar – I find it relevant that the posh boarding school is the first time he ever had friends his own age, but the book doesn't lean much on those friendships to appeal to why he was miserable when he was taken away.

It is indeed pretty bleak that his one bit of agency is discovering that he enjoys the path that the adults have set for him. :(