Something like you are looking for would probably be possible if your TV supported Bluetooth. While I was experimenting with what was possible, I connected my TV to my Echo Dot using Bluetooth and found that the volume from the TV was turned down when I said "Alexa". You would basically have your TV connected to the Echo Dot via Bluetooth and the Echo Dot connected via a cable to the speaker.
However, this wouldn't be possible if the TV doesn't support Bluetooth (although you can buy Bluetooth transmitters which allow you to add Bluetooth support to things like televisions which don't support it) and it is probably a bit fiddly to use every day (you would sometimes need to say "Alexa, connect my phone" to get it to reconnect to the television).