maemo.org - Talk - [Announce][Dirty Hack][N900/N9] Mee42 v0.3.1

maemo.org - Talk (https://talk.maemo.org/index.php)

- Applications (https://talk.maemo.org/forumdisplay.php?f=41)

[Announce][Dirty Hack][N900/N9] Mee42 v0.3.1 - Poor man's Siri resp. Google Now

Some months ago a friend was really impressed by Google Now, a kind of answering machine on Android, more or less equivalent to Apple's Siri or whatever it is called. Looking at some demonstration videos on youtube i saw everything i needed to get the below shell script finished. By the way, it works pretty well for me. Feel free to use it, modify and optimize it, print and eat it to become one with the code:)

About Mee42
Essentially the shell script works as follows: Record audio from mic & convert to flac format -> Google Voice API for speech recognition -> do some text manipulations here and there -> get an answer from wiki.answers.com -> read the answer using espeak.

Security aspects
Sending a text phrase to answers.com is one thing, sending your individual voice samples to Google's cloud is another thing. Just be aware what you are doing. It's your data.

About GUI discussions, N900/N9 versions etc
My aim was to share the script as a proof of concept so that the community can profit from it as soon as possible. Taixzo who develops Saera, a Siri clone with GUI which is available for the N900, integrates the essential ideas into his app. The best thing to do now is to support him by voting for Saera in 2012's Coding competition. With a N9/N950 in his hands the chances are even better to see a Harmattan port of Saera.

Changelog

Changelog version 0.3.1 (25.09.2012), mee42_v0.3.1.sh:
- Fixed potential issue when N9's GSM/UMTS network is used: Now the answer file ${file_prefix}.html is handled correctly when line breaks are missing.
- Minor code changes like 'about' information.

Changelog version 0.3 (24.09.2012), mee42_v0.3.sh:
- Added voice feedback before/after recording (thanks to bingomion for requesting this)
- Added voice command to exit the loop: 'exit', 'quit' or 'stop' exits the loop now (again thanks to bingomion for requesting this)

Changelog version 0.2 (23.09.2012), mee42_v0.2.sh:
- Renamed 42^infinity to Mee42:)
- Runs now on the N9:)
- Replaced arecord + flac with gst-launch.
- Get rid of vi(m) command.
- Several minor changes for code readability.

Initial release from 23.09.2012 can be found here mee42_v0.1.sh

Output of a single (!) terminal session

Code:

~/bin $ ./mee42_v0.3.1.sh

-----------------------------------------------------------------------------

Mee42 is a shell script which records your voice and tries to answer your

question using Google Voice API for speech recognition, answers.com for db of

answers and espeak for speech synthesis. CopyStraightForward blueslee@tmo.



Recording sound input.................

Speech2txt via Google Voice...........who is Michael Jordan

Getting answer from answers.com.......He is a famous basketball player from the NBA (NationalBasketball Association). He is known for his glory days with the Chicago Bulls.

Txt2speech via espeak.................



Recording sound input.................

Speech2txt via Google Voice...........the capital of Tonga

Getting answer from answers.com.......Nuku'alofa is the capital of Tonga. .

Txt2speech via espeak.................



Recording sound input.................

Speech2txt via Google Voice...........what is the biggest prime number

Getting answer from answers.com.......There is no biggest prime number. The largest prime known at present is 243112609 - 1, a number with nearly 13 million decimal digits.

Txt2speech via espeak.................



Recording sound input.................

Speech2txt via Google Voice...........what is the best smartphone

Getting answer from answers.com.......Iphone_and_Blackberry:D

Txt2speech via espeak.................



Recording sound input.................

Speech2txt via Google Voice...........#### you

Getting answer from answers.com.......Something went wrong.

Txt2speech via espeak.................



Recording sound input.................

Speech2txt via Google Voice...........what is 42

Getting answer from answers.com.......42 is the answer to life, the universe, and everything. (From the book Hitch hikers guide to the galaxy.) EDIT: aww you beat me to it...

Txt2speech via espeak.................



Recording sound input.................

Speech2txt via Google Voice...........exit

Shell Script Version V0.3.1

Code:



#!/bin/sh

#

# Mee42 - Siri/Google Now for the poor man ;-)  

#

# CopyStraightForward blueslee@tmo

#

# What you need: gst-launch or alternatively arecord + flac, wget, espeak



# Choose your language here, only English is supported yet  

# (html structure of answers from *.answers.com are different)

lang="en"

if [ "${lang}" = 'en' ]; then 

   lang_flag='wiki'

else 

   lang_flag=${lang}

fi



# Maximum speech recording time in sec

speech_rec_time=4                                          #use for arecord

speech_num_buffers="$(( 50 * ${speech_rec_time} ))"          #used for gstreamer 



# Default sample rate for flac. Use 48000 if you are using arecord + flac 

# as flac's --sample-rate=16000 does not work for me  

sample_rate=16000



# Just a shortcut for Google's Voice API Url

api_url='http://www.google.com/speech-api/v1/recognize?lang=${lang}-${lang}\

         &client=chromium'



# Temporary files are named as below with the corresponding suffix

file_prefix='file_temp'

rm -f ${file_prefix}* 





about()

{

cat << EOF

-----------------------------------------------------------------------------

Mee42 is a shell script which records your voice and tries to answer your 

question using Google Voice API for speech recognition, answers.com for db of 

answers and espeak for speech synthesis. CopyStraightForward blueslee@tmo.



EOF

}





# Record voice via gst-launch, alternatively you can use arecord and flac

record_voice()

{

    gst-launch-0.10 pulsesrc num-buffers=${speech_num_buffers} ! audioconvert\

     ! audioresample ! audiorate ! audio/x-raw-int,rate=${sample_rate} ! flacenc\

     ! filesink location=${file_prefix}.flac 1>/dev/null 2>&1



    #arecord -f dat -d ${speech_rec_time} ${file_prefix}.wav 1>/dev/null 2>&1

    #flac --compression-level-5 ${file_prefix}.wav 1>/dev/null 2>&1

    #Remark: sox on N900 doesnt work as below i.e. has no flac support 

    #sox ${file_prefix}.wav ${file_prefix}.flac rate 16k gain -n -5 silence 1 5 2%

}





# Speech2Txt uses Google Voice API

speech2txt()

{

    wget -q -U "Mozilla/5.0" --post-file ${file_prefix}.flac\

     --header="Content-Type: audio/x-flac; rate=${sample_rate}" -O\

     - ${api_url} > ${file_prefix}.ret

    cat ${file_prefix}.ret | sed 's/.*utterance":"//'\

        | sed 's/","confidence.*//' > ${file_prefix}.txt

}





# Get answer from answers.com

get_answer()

{

    # We are replacing ' ' by '_' as a typical request is handled in the http 

    # header, for instance http://wiki.answers.com/Q/who_is_michael_jordan

    sed -e 's/ /_/g' ${file_prefix}.txt > ${file_prefix}2.txt



    request_url="http://${lang_flag}.answers.com/Q/"`cat ${file_prefix}2.txt`

    wget ${request_url} --output-document=${file_prefix}.html 1>/dev/null 2>&1

   

    # The first occurance of 'description' contains our answer

    grep 'og:description' ${file_prefix}.html | head -1\

        | sed -e 's/.*og:description//' | cut -d"\"" -f3,3 > ${file_prefix}.txt

}





# Sanity check if something went wrong, i.e. we have no or wrong answer

error_handling()

{

    errorcheck=`cat ${file_prefix}.txt | cut -c1-4`

    if [ "${errorcheck}" = "http" -o "${errorcheck}" = "" ]; then

       echo 'Something went wrong.' > ${file_prefix}.txt

    fi

}





# Txt2speech via espeak

txt2speech()

{

    espeak -v${lang} -f ${file_prefix}.txt 1>/dev/null 2>&1

}





# Cleanup of temporary files

cleanup()

{

    rm -f ${file_prefix}* 

}





clear

about



while true; 

do         

    espeak -v${lang} "Ask your question or say exit." 1>/dev/null 2>&1

    printf 'Recording sound input.................'

    record_voice

    espeak -v${lang} "Thank you." 1>/dev/null 2>&1

    sleep 1

    echo



    printf 'Speech2txt via Google Voice...........'

    speech2txt; cat ${file_prefix}.txt



    # Leave the while loop if users says 'exit' || 'quit' || 'stop'

    exit_code=`cat ${file_prefix}.txt`

    if [ "${exit_code}" = "exit" -o "${exit_code}" = "quit"\

                                 -o "${exit_code}" = "stop" ]; then

        cleanup

        exit 0

    fi



    

    printf 'Getting answer from answers.com.......'   

    get_answer

    error_handling 

    cat ${file_prefix}.txt



    printf 'Txt2speech via espeak.................' 

    txt2speech

    echo; echo

   

    cleanup

done

Installation

N9
You will need to install the debian packages mentioned in the script, i hope the below list is complete:
- wget should be available from Nokia's default repository, install it via 'apt-get install wget'
- Download and install gstreamer0.10-tools from Harmattan's repository using 'dpkg -i' on your device.
- Download and install espeak-data, libespeak and espeak_1.46.11 from Rzr's repository using 'dpkg -i' on your device.
- Save the shell script to your N9 as mee42.sh, do a 'chmod u+x mee42.sh' and run it via './mee42.sh'.

N900
- Should be easier as all debian packages should be available on extras. Note that you need to install in addition gstreamer0.10-flac

Desktop
- Try to switch to alsasrc for recording with gst-launch if pulsesrc is not available.

As always have fun.

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

cool yaaaaaaaaaaaaaar !
if some how this could be a part of saera ?

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

This looks hard for a noob like me

so what should i do? should i wait fr some detailed or step by step instruction?

or some one is going to make deb for it?

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

so we need an icon for n9, and deb file too, :D

awesome

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

What exactly does this line do:

Code:

vim -c ':/description' -c ':.-1' -c ":1,. d" -c ":.+1" -c ":.,$ d" -c ":wq!" ${file_prefix}.html

I'm trying to implement this in Python.

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

Quote:

Originally Posted by taixzo (Post 1270849)

What exactly does this line do:

Code:

vim -c ':/description' -c ':.-1' -c ":1,. d" -c ":.+1" -c ":.,$ d" -c ":wq!" ${file_prefix}.html

I'm trying to implement this in Python.

-c ':/description' => find 'description'
-c ':.-1' => goto previous line
-c ":1,. d" => delete all lines from beginning to current line (1 line before 'description')
-c ":.+1" => goto next line
-c ":.,$ d" => delete all lines all to the end
-c ":wq!" => save

basically it extracts the expected answer from the result HTML using 'description' as keyword

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

Quote:

Originally Posted by taixzo (Post 1270849)

What exactly does this line do:

Code:

vim -c ':/description' -c ':.-1' -c ":1,. d" -c ":.+1" -c ":.,$ d" -c ":wq!" ${file_prefix}.html

I'm trying to implement this in Python.

Essentially, it searches for the first occurence of a line containing "description" in ${file_prefix}.html and saves the file. Replacing it by

Code:

grep "description" ${file_prefix}.html | head -1

makes the usage of vi(m) redundant and it should be also much faster. For details see the answer of figaro.

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

Clever little script!!

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

have you tested it on harmattan ? let me suggest to rename it too something less enigmatic , ie : google-speech-ui ..

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

Quote:

Originally Posted by www.rzr.online.fr (Post 1271087)

have you tested it on harmattan ? let me suggest to rename it too something less enigmatic , ie : google-speech-ui ..

You are right concerning the name. As there will be a N9 version i renamed it to Mee42:)

Taixzo the author of Saera will hopefully integrate the idea into his app so that we can all profit from it. Unfortunately, for Harmattan some tools like sox, flac etc. are missing. I will try to switch to alternatives.

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

Here we go

Changelog version 0.2:
- Renamed 42^infinity to Mee42:)
- Runs now on the N9:)
- Replaced arecord + flac with gst-launch.
- Get rid of vi(m) command.
- Several minor changes for code readability.

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

Quote:

Originally Posted by BluesLee (Post 1271166)

Here we go

Changelog version 0.2:
- Renamed 42^infinity to Mee42:)
- Runs now on the N9:)
- Replaced arecord + flac with gst-launch.
- Get rid of vi(m) command.
- Several minor changes for code readability.

script sounds cool
however, the recording part doesn't really work for some reason on N9 :)

Re: [Dirty Hack][N900 / maybe N9?] 42^infinity - Poor man's Siri resp. Google Now

Quote:

Originally Posted by Jeffrey04 (Post 1271181)

script sounds cool
however, the recording part doesn't really work for some reason on N9 :)

Can you please post the output of

Code:

gst-launch-0.10 pulsesrc num-buffers=200 ! audioconvert !  audioresample ! audiorate ! audio/x-raw-int,rate=16000 ! flacenc ! filesink  location=file_temp.flac

If a file is generated play it with the default media player (for instance using filebox), mplayer did not do the job well.

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

I like v2 (VIM was 6megs wow lol)
two suggestions:
1.play TTS 'Ask your questions?' before starting recording, play TTS 'thanks' after recording
2.voice command for stopping the loop, exit or stop, quite etc

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

There any screenshot ? And i want to ask a kind of stupid question .
this app is like siri on iphone ?

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by Rabah_vip (Post 1271261)

There any screenshot ? And i want to ask a kind of stupid question .
this app is like siri on iphone ?

This is not an app, just a script which runs in XTerm.
I've written a Siri-like app, which will use this after the coding competition is over.

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

give it gui please, for harmattan, :)

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

dpk not found how to fix it please ?

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by Rabah_vip (Post 1271408)

dpk not found how to fix it please ?

dpkg...

10chars

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

thank you but there is a problem could not open the debian archive
compilation failed in require

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by Rabah_vip (Post 1271411)

thank you but there is a problem could not open the debian archive
compilation failed in require

It means you copied the .deb file to a different directory from the path specified in the dpkg -i command....Why not download all the files needed to one folder say /home/user/MyDocs/Downloads/Mee42/

Then as root just do:

Code:

dpkg -i /home/user/MyDocs/Downloads/Mee42/*.deb

Post your full output here otherwise...

Edit: Before you ask, yes I've tested and it works so it should work for you too :)

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

someone can help me with this how to do it :
Save the shell script to your N9 as mee42.sh, do a 'chmod u+x mee42.sh' and run it via './mee42.sh'.

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by Rabah_vip (Post 1271421)

someone can help me with this how to do it :
Save the shell script to your N9 as mee42.sh, do a 'chmod u+x mee42.sh' and run it via './mee42.sh'.

Extract mee42.sh from my attachment...Save it to say /home/user/

Then as root do:

Code:

chmod u+x /home/user/mee42.sh

To run the script do:

Code:

sh /home/user/mee42.sh

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Hi!

I've installed it, but the output is everytime "Sorry, I did not get your question".

I've also tried with the questions in the first post...

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by Gandan (Post 1271437)

Hi!

I've installed it, but the output is everytime "Sorry, I did not get your question".

I've also tried with the questions in the first post...

It hates your voice :p

Jokes aside, BluesLee has said this is a "Dirty Hack"...Don't expect much accuracies...The dev of Saera posted a page back that he will incorporate this into Saera so maybe a Saera for n9/900 would do the job for you...

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by thedead1440 (Post 1271438)

LOL

I hope... I've bought Speechcommand on Nokia store, but it isn't the same thing...

Thanks! :)

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by thedead1440 (Post 1271438)

I've already made a development build with this integrated; hopefully I will be able to make an N9 version soon.

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by taixzo (Post 1271465)

I've already made a development build with this integrated; hopefully I will be able to make an N9 version soon.

Would be very much appreciated :)

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

what about n900? is anyone building deb file for it? i mean any gui for it?

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by nicholes (Post 1271517)

what about n900? is anyone building deb file for it? i mean any gui for it?

Did you read anything at all?

The dev for Saera [a n900 app], taixzo, clearly stated that he has incorporated this into it and waiting for the Coding Competition to end before he can upload it...

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by nicholes (Post 1271517)

what about n900? is anyone building deb file for it? i mean any gui for it?

I think you mean Saera by taixzo

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by thedead1440 (Post 1271526)

Did you read anything at all?

The dev for Saera [a n900 app], taixzo, clearly stated that he has incorporated this into it and waiting for the Coding Competition to end before he can upload it...

i read that post. my english is not good.this is what now i hv understood

this post says that it will be available for N9

and This post says it will be for N900

??????

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Ok I'll explain it to you in simple English then :)

Saera is at this moment a n900-only app which is participating in the 2012 Coding Competition (link for which you can find on the top right side of your page)

As per the rules of the Competition, only bug fix updates are allowed till voting closes i.e. NO new features are allowed.

Its pretty obvious that while a developer won't release any new features; he will still work on them right?

Hence, taixzo has in his own personal, private Saera build added this Mee42 script.

After the end of the Voting for the 2012 Coding Competition, he will be releasing a updated n900 version of Saera with Mee42 integrated. He will also attempt to build a similar version for the n9 which at this point doesn't have Saera.

Profit?

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by nicholes (Post 1271538)

i read that post. my english is not good.this is what now i hv understood

this post says that it will be available for N9

and This post says it will be for N900

??????

Saera is already available on the N900. I hope to bring it to the N9 as well.

Re: [Announce][Dirty Hack][N900 / N9] Mee42 v0.2 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by thedead1440 (Post 1271544)

i got it !! :)

Thanks a lot

Re: [Announce][Dirty Hack][N900/N9] Mee42 v0.3 - Poor man's Siri resp. Google Now

I added some information concerning GUI development etc on the initial post:) Here are some minor but helpful improvements:

Changelog version 0.3:
- Added voice feedback before/after recording (thanks to bingomion for requesting this)
- Added voice command to exit the loop: 'exit', 'quit' or 'stop' exits the loop now (again thanks to bingomion for requesting this)

Re: [Announce][Dirty Hack][N900/N9] Mee42 v0.3 - Poor man's Siri resp. Google Now

i cannot run chmod u+x
im rooted
it says "chmod: mee42.sh: Operation not permitted"
:confused: :confused: :confused: :confused:

Re: [Announce][Dirty Hack][N900/N9] Mee42 v0.3 - Poor man's Siri resp. Google Now

Quote:

Originally Posted by ueharaf (Post 1271801)

i cannot run chmod u+x
im rooted
it says "chmod: mee42.sh: Operation not permitted"
:confused: :confused: :confused: :confused:

Where did you copy mee42.sh to? if you didn't copy it to root directory but to /home/user/ then try without root...