In order to set up this functionality on your Alexas, it appears you do need a smart cell phone. (The little "chat balloon" widget at the center bottom of the Alexa app screen doesn't appear if you are not running the app on a cell phone.) However, I believe what the app really needs is your phone list -- your phone's address book -- not your cell phone number. It needs this list in order determine to with whom to place a call: When you say "Alexa, call Ted", it needs to look up the "Ted" entry in your phone book and compare the phone numbers you have for "Ted" to see if one of these is associated with another Alexa account Amazon has on file (presumably Ted's).
You do not need to associate your cell phone number with your Alexa account however. All you need is to assign it a phone number that is capable of receiving texts (for verification purposes). I associated a Google Voice number I give out to friends and family with my Alexa app and it is working fine. (After all, it is already in all my friends' and family's address books.)
So, bottom line: You do need a smart phone with an address book that contains entries for everyone you'd like to call using the Alexa app. And, you need a phone number that can receive texts, and that you are willing to give out to people who you want to talk to via Alexa. But, that doesn't have to be the phone number (if any) that is activated on your smart phone.