Post

API for twiqs.nl

In TwiNL on 24/06/2014 by admin

An API (Application Programming Interface) is an interface which can be used by computer programs to communicate with other programs. There are several ways for computer programs to start search queries on twiqs.nl and retrieve the results. In this blogpost, we give an example which involves the command curl which is available on Macs and computers with the Linux operating system.

Suppose you would like to know what words were used in Dutch tweets that contained the word chili (Dutch for the country Chile) and were written on Monday 23 June 2014, the day that the Dutch soccer team played against Chile in the World Cup. At twiqs.nl you can obtain a word cloud by setting the correct search options and pressing the search button. However, you want a program to start the search process and retrieve the results. The following command will achieve this:

   curl -o output.txt 'http://145.100.59.92/cgi-bin/twitter?SEARCH=chili&
         DATE=2014062300-2014062323&SHOWCLOUD&DOWNLOAD'
 

(the command is supposed to be written on one line) The command will store its output in the file output.txt. It accesses the search program on twiqs.nl with four parameters written with capital characters. SEARCH holds the query, in this case the word chili. DATE states in what time frame should be searched: from 0:00 until 23:59 on the 23 of June 2014. It contains the start date and time and the end date and time, both in the format: year, month, day and hour. SHOWCLOUD and DOWNLOAD specify that the word cloud results are requested and that we want them to be downloaded.

When you run this command like this it will finish within a few seconds but probably not give you any results. The reason for this is that results can only be presented after the search command on the web server is finished, which might take a few minutes. The first time you run the command, the web server will start the search process but no results will be returned. You may repeat the command as often as you want. No results will be returned until they are available.

This means that you will need to run the same command several times until it generates output. If the output file does not exist already, it will only be created when the search process is finished. If a search process has no results, the output file will just contain the heading for results of these queries: #token #t-score #frequency-token-in-selection #tokens-in-selection #types-in-selection #frequency-token-in-day #tokens-in-day #types-in-day.

People with a registered account on twiqs.nl might want to access the website automatically with the same access level as when being logged in. In order to achieve this, you should supply a cookie with the command, with the curl option --cookie 'cookie=ReplaceWithCookieValue'. A valid cookie value can be obtained by logging into the website as usual and getting the cookie value from the browser, for example in Firefox: Preferences -> Privacy -> Show Cookies -> drop down menu 145.100.59.92 -> 145.100.59.92 cookie -> Content. Cookies are associated with the client machine they were generated for so you can not use them on other machines. Cookies do not expire on twiqs.nl, so you can use the same cookie forever.

(note: the ip adress of twiqs.nl mentioned in this post may change in the future)

Leave a Reply