How to use the Google Suggest API to come up with topics for Answers

Can you suggest how this script can be modified to use 'past month' or 'past 2 weeks' to limit the search. Also, I am interested in how to enter a date range, for example, 1998 through 2008. Can the results be sorted by the numerical result? How do you direct the output to a text file?

Lots of requests I know but any help would be appreciated.

Thanks,
George

Hi, georgerix. I think the suggest API is pretty limited -- it always returns just 10 most recent results (unless there are other query parameters that I don't know about.) So, you can't really "automatically" get what you want. Obviously, you could just run the script every 2 weeks and start tracking the results over time, but that's probably not what you're after.

The closest equivalent thing I can think of would be to use the search api -- it won't give you what people are searching for, but it will at least tell you the top things people find for a given phrase. I wrote a piece on it called Simplify business research with Google Ajax Search API, but there are a lot of ther resources out there. That API has a wide set of parameter you can tweak.

Redirecting to a text file is pretty easy. Assuming you've saved the script using the name "suggest.py," you can drop into a terminal and enter this command:

python suggest.api > out.txt

This will redirect the output into a tab-delimited text file. From there, you can import it into excel and do the sorting and searching.

Hope this helps!

ps -- to run this, you'll need a Python interpreter. If you're on a Mac, it's built-in, but on a PC, you might need to install one. ActivePython, from ActiveState, is a pretty good all around choice because they've done a great job making a nice, installable system. Good luck!

Andrew,

Thanks for the help, much appreciated. I ran the script in Automator on the Mac (Snow Leopard) to play with it rather than from the command-line. I will need to learn a little more before I can make use of the APIs. Thanks for the suggestions.

One last thing, can the output be directed to a text file from within the script?

Thanks again,
George

George --

You can use basic file I/O to write the results out -- try this modified version should do it, which (should!) save the results to a file called "out.txt" on your Desktop.

Andrew

from sgmllib import SGMLParser
import urllib2
import urllib

# Define the class that will parse the suggestion XML
class PullSuggestions(SGMLParser):

   def reset(self):
      SGMLParser.reset(self)
      self.suggestions = []
      self.queries = []

   def start_suggestion(self, attrs):
      for a in attrs:
         if a[0] == 'data': self.suggestions.append(a[1])

   def start_num_queries(self, attrs):
      for a in attrs:
         if a[0] == 'int': self.queries.append(a[1])


# -----------------------------------------------------------
# Enter your phrase here.  Be sure to leave the %s at the end!
# -----------------------------------------------------------
base_query = "how do i %s"  #This is the base query
f = open( '~/Desktop/out.txt','w')

alphabet = "abcdefghijklmnopqrstuvwxyz"
for letter in alphabet:
   q = base_query % letter;
   query = urllib.urlencode({'q' : q})
   url = "http://google.com/complete/search?output=toolbar&%s" % query

   res = urllib2.urlopen(url)
   parser = PullSuggestions()
   parser.feed(res.read())
   parser.close()

   for i in range(0,len(parser.suggestions)):
      f.write("%s\t%s\n" % (parser.suggestions[i], parser.queries[i]))

f.close()

I have to say, your script is great! However, I just wonder how can I change the key word by sending it by get method? For example, it shows the result when I type /?q=how+do+i

thanks very much!

Hi, snail000. I'm not quite sure I understand your question, but here's a shot. If you want to change the keyword, you can change the string in the base_query variable. (It comes just before the big loop that does the work.) If that's not your question, can you clarify?

Nice work.

The script exits when it encounts a string that doesn't have any suggestions.

eg : if there is no suggestion for "how do i q" the output stops there.
Is there any way to fix that?