►►► Alexa does searches, returns images on screen: Video ◄◄◄

[Jez]

As promised, here's a video of Alexa controlling and initiating web searches using Google, YouTube, Wiki, Bing and more all simultaneously. Also PC control through 'To Do' lists. Can also fire off local files e.g. Photo albums, playlists etc. Much more to come.

Please comment and give feedback.

https://youtu.be/X-dOkuFGWSc

Jez.

[mike27oct]

Impressive.

Since Alexa is part of Fire TV, can all (or part of) this be displayed via a Fire TV, or is it viewable only on a PC (or in a "Bat Cave")?

It seems many of the questions could have been answered by plain ol' Alexa (minus the visuals). What percent of the questions you asked could not be answered by plain vanilla Alexa?  The addition of visuals is a big plus.

Keep us posted.

[Jez]

Hi, Thanks for the feedback, that's what this is all about at the moment.

It's true that the questions I asked could have been asked without the visuals, but the visuals add so much more and offer so many other tangents once they come up. I added 4 browsers on purpose so that many variations of the same question & answers could be displayed. The fact that you can tweak each individual browser to accept the question + a little bit extra can show so much more detail .... something you would rarely bother doing manually ..... BatCave stuff.

(Browser 1 Bing) Who is + (Browser  2 Yahoo)  where were they born + (Browser 3 Google) Where do they work + (Browser 4 Wiki) achievements etc ... that one question gives a lot of reading material.

More importantly, Alexa may not have understood the questions but the browsers did and they still return their own findings. It would be vastly improved by having Amazon offer alternative 'I don't know the answer to that' to instead simply repeat what was asked with a paused 'errrrrm working on it' on the end. Because the browsers will still find it. Alexa is acting as a keyboard .... and that's about it. Ask the question in the right way and Alexa will offer a contribution to the answer .... but doesn't matter if she (it) doesn't.

Home automation is fast becoming an automated light-switch, I think we can get a lot more than that out of Alexa, I've never seen such brilliant voice recognition before ..... especially from the other side of the room. Currently have 2 dots and one echo.

What do you need to use the application.

A single standard PC, Win7 - to - Win10 etc.
 
Each PC running the application would offer up to 4 Browsers where any or all of these could be placed on individual monitors in full screen.

Install the application on more than 1 pc will add (4) browsers each time with each PC having potentially a different search engine configuration for every question asked. All PC's would work in sync without any setting up whatsoever, even if they're in different rooms.

Also, the search engines can change based upon the question,
Q: What Wine goes with Beef,
Could result in a search including Asda, Tesco, Sainburys and a wine merchant for alternatives and prices.

Q:Who sang ...
Could result in a list of YouTube results, Facebook fan club, Relevant Images, Next live venues with dates. etc.

Sorry ... you can probably see the passion already.


Jez.

[mike27oct]

Yep, Jez, the visuals are the BIG plus of this.  I was just wondering if Alexa got stumped many times, but the browsers "got it".

So, is this a program to install on a PC?  When do we get to play with it ourselves?

[Jez]

I own all of the stuff I write .... so potentially I could put a self-installer on line tomorrow. I really wanted feedback first ... add a website ... get some nice people involved in trying and testing it ... selling it to millions.

Nothing too ambitious you see.

This forum is great for this and I could quite happily drop several members a free copy in return for constructive feedback.


Jez.

[mike27oct]

Jez, since I have experience as a beta tester for a major electronic gadget company, I will keep an eye out for a PM from you.   
--Mike

[jim1590]

I would be happy to volunteer my time to test some things out.

Also my wife is not to tech savvy so to speak. I can have her test things out as well as someone who isn't all computer geek.

[Jez]

Hi Jim,

Thanks for the offer of testing, I look forward to your feedback. Should be ready next week, finishing off the basics of a new site for it http://www.addtopad.com and adding the ability to launch any pc application to a desired spot on the screen along with groups of applications fired up from one command. e.g. Do Banking ...., Excel, Bank login sites, Calculator etc

Jez.

[mike27oct]

Jez,

As I offered above, I would like to test your new Alexa websearcher. too.
Mike

[staza]

I want to be a tester too...if it is possible. I have a lot of experience.

[brantome]

Watching with interest. Integration with Alexa Smart Home devices I'd hope would be there through anyway thru Alexa voice control.
Had a quick look at the website - £20 sounds not bad for the initial purchase, but I really don't like subscription models for software for ongoing support/ facilities. Small incremental versions should be free, with major updates chargeable and optional. A price for lifetime upgrades is perhaps another option I'd consider.

[meep]

Hi

In another thread, you mentioned that you could identify the echo to which the command was spoken and throw results to a nearby screen.

Would you mind expanding on that a little and how it can be set up with this app?

Echos don't have any deviceID sent to skills so I'm assuming you have some network sniffing going on?

Looks great, btw!

Cheers

[Jez]

If you look at your History page in the Alexa app you will notice that whatever was heard is logged but just below that is the device that heard it.

The application simply reads what was heard and also reads the device name. I don't therefore need any device ID's we just need to create a set of actions depending on what is asked of Alexa. If you said 'Alexa ... Dinnertime' you might want all monitors to show a big dinner gong being hit. If you ask 'Alexa ... Backgammon' then you might want that to go to just the games table driven by a Raspberry Pi.

First step would be to list your Echo/Dot/Tap names in the application connections file,
Dot-1
Dot-2
Echo-1

If you add Phrases (To listen for) below the device name then you're saying that you only want the HTML response if it was heard from that device, e.g,

In the next line, the application is listening for 'Backgammon' if it reads it, the response will be sent to any monitor that has ?S=5 in its URL.

Phrase:Backgammon
NEWURL-5:http://addtopad.com/images/games_table/games.php?G=backgammon

The same Phrase but listed under a device name means that it should only be sent if it was heard from a particular Echo/Dot/Tab.

Dot-2
Phrase:Backgammon
NEWURL-5:http://addtopad.com/images/games_table/games.php?G=backgammon
Phrase:Snakes and ladders
NEWURL-5:http://addtopad.com/images/games_table/games.php?G=snakes

The URL is then sent to the monitor/device where it is then launched in a separate 'Named' tab. So If you then said 'Alexa .... Snakes and ladders' it will send the new URL, launch it into the same named tab overwriting the previous HTML contents.

This isn't released yet but should only be days away. Got it to now launch PC apps too. e.g. Say 'Alexa .... Banking' opens any number of PC apps simultaneously and positions and sizes each on the screen wherever you want.
Banks webpage
Excel spreadsheet
Folder where you save your bank downloads
etc.

[Jez]

Another one, try this URL in a browser,
http://www.beatone.co.uk/cocktails

We now put a monitor and Raspberry Pi next to our bar. We'll name the monitor number '7' We want Alexa to send responses just to this monitor when we ask 'Alexa .... Cocktails'

Connections file entry,

Dot-3
Phrase:Cocktails
NEWURL-7:http://www.beatone.co.uk/cocktails/[SEARCH-]

So what happens:
You just said 'Alexa ... Cocktails' so the URL is sent to that monitor (7) but only if you said 'Alexa ... Cocktails' into Dot-3
When the page loads it shows lots of Cocktails so you could now say 'Alexa .... Cocktails June Bug' and it will send the same URL but with the '[Search-]' bit replaced with 'June Bug' and will load the recipe for that Cocktail.

The hyphen in the '[Search-]' tells the application to replace spaces with hyphens. could be '+' '%20' or nothing.