I supposed I should have specified that the majority of the Pokemon don't effect the story. But the legendaries and starters are just what around 20 Pokemon at most that are tied with the story. For the most part, the Pokemon don't really affect much of the scope of the game. In terms of gameplay, I'm pretty sure they design them, create their moveset/stats/capture rate and just put them in whatever parts of the map and trainers they want to have them. They usually do some research into whatever real world location they want to use and then form the basis of the region's Pokemon based on the wildlife and culture/lifestyle of the place.
I do however I agree that it would be nice to have games have more emphasis on the Trainer's adventure. These games would be perfect for in-between generations, but it doesn't seem like GF has an intention to do that. Let's Go would have been an ideal way to do this by maybe having Lillie's Kanto journey that was hinted in S/M or some sort of sequel set in Kanto, but it seems they just wanted to do a 1:1 Pokemon Yellow remake for the casual fans.
In terms of limited Pokemon, each game limits them . No games(except the first Gen ofc) has all of the Pokemon between the two versions. Ultra Sun/Moon for example only had like 400 out of the 807 total Pokemon. There is also no National Dex in the games, so GF doesn't expect you to actually catch em all to get all the game's rewards anymore. Only the competitive players have to deal with balance and gameplay that comes with having 800+ Pokemon (even then they limit the Legendaries in most cases)