Thanks to a few inspirations from the previous posts, this is what worked for me. (I have tried resetting the lightbulbs/echo plus in the past but still struggled)
1. Reset the Echo Plus.
2. Log out of the Alexa App
3. Once the Echo Plus is ready to be set up, log in to the Alexa App
4. Once set up, manually reset each of those hue lightbulbs
5. That should discover them one by one.