Commit 7544044b authored by Dr. Daniel Diaz Sánchez's avatar Dr. Daniel Diaz Sánchez
Browse files

Add new file

parents
Este proyecto se puede clonar. Lee las instrucciones al final de la página.
# Acceso a interfaces REST. Descarga masiva de Tweets
### Aplicaciones y librerías a utilizar
* Eclipse (podéis usar el que descargásteis el primer día)
* [Unirest](http://unirest.io/java.html): librería de peticiones HTTP (más adelante veremos como descargarla)
* [Commons](https://commons.apache.org/): librerías de funciones auxiliares (más adelante veremos como descargarla)
* [Google-Gson](https://github.com/google/gson): librería de manipulación de JSON (más adelante veremos como descargarla)
Será necesaria una cuenta de twitter
## Crear una aplicación en Twitter
Si no tenías twitter, ha llegado el momento de tenerlo, hazte una cuenta. Si ya tienes una, haz login en twitter.
Tuvieras o no, consulta #uc3m_cd que es la cuenta de la asignatura aunque se usa poco ;)
Vamos a descargar tweets. Para ello, debes dar de alta una aplicación. Ve a la [gestión de apps de twitter](https://apps.twitter.com/) y crea una aplicación.
<img src="https://gitlab.pervasive.it.uc3m.es/distributed-computing-assignements/2-restAPIs-unirest-twitter/raw/master/img/image008.png" width="500px"/>
Rellena el formulario (no rellenes la CallBackURL)
<img src="https://gitlab.pervasive.it.uc3m.es/distributed-computing-assignements/2-restAPIs-unirest-twitter/raw/master/img/image010.png" width="500px"/>
Genera los tokens hasta que tengas los siguientes datos (además del consumer key etc). Si no los generas todos más adelante verás que hacen falta. No hay problema, siempre puedes volver atrás.
<img src="https://gitlab.pervasive.it.uc3m.es/distributed-computing-assignements/2-restAPIs-unirest-twitter/raw/master/img/image012.png" width="500px"/>
## Descarga de tweets con la ayuda de desarrollador de Twitter
Ahora vamos a usar un API de verdad. Está documentado correctamente, de hecho, el de twitter, está muy bien comentado y tiene utilidades que veremos. Ve a la página:
https://dev.twitter.com/rest/public
Observa como todo lo que se puede hacer con twitter tiene su correspondiente entrada en el API.
Vamos a echar un vistazo a la función para obtener tweets que es `GET statuses/user_timeline`
Para ello, ve a la página https://dev.twitter.com/rest/reference/get/statuses/user_timeline
Twitter cuida mucho su interfaz, y ayuda a conservarlo y entenderlo. Pero dado su éxito, ha prohibido el acceso anónimo. Por lo que es necesario usar OAuth (usando los tokens que hemos generado) para acceder a cada función del API rest. Para usar OAuth, es necesario calcular una firma y añadirla a la cabecera Authorization de HTTP.
Para que puedas probar de forma rápida el API, Twitter proporciona un mecanismo para generar esas cabeceras y probarlo.
Si lees con atención la página (y estás logeado en twitter - si no, no funcionará), llegarás a un sitio en el que verás un desplegable con tus aplicaciones:
<img src="https://gitlab.pervasive.it.uc3m.es/distributed-computing-assignements/2-restAPIs-unirest-twitter/raw/master/img/image014.png" width="500px"/>
Si seleccionamos nuestra aplicación en el desplegable "Select one of your apps", nos redirige a otra página, en la que calcula por nosotros la susodicha cabecera:
<img src="https://gitlab.pervasive.it.uc3m.es/distributed-computing-assignements/2-restAPIs-unirest-twitter/raw/master/img/image016.png" width="500px"/>
Si pulsamos en Get OAuth Signature, obtendremos una cabecera Authorization adecuada:
<img src="https://gitlab.pervasive.it.uc3m.es/distributed-computing-assignements/2-restAPIs-unirest-twitter/raw/master/img/image018.png" width="500px"/>
Utiliza la última linea (curlCommand) para directamente en la consola para probar la petición. Copia y pega.
Entonces sabemos que podemos consultar esa información de la siguiente manera:
```
GET 1.1/statuses/user_timeline.json?count=2&screen_name=twitterapi HTTP/1.1
Host: api.twitter.com
Authorization: ...
```
Esto nos dará un JSON de este tipo:
```json
[{"retweeted_status":{"contributors":null,"text":"We have disabled SSLv3 protocol support in response to the vulnerability published today. You may need to update your browser to use Twitter","geo":null,"retweeted":false,"in_reply_to_screen_name":null,"truncated":false,"lang":"en","entities":{"symbols":[],"urls":[],"hashtags":[],"user_mentions":[]},"in_reply_to_status_id_str":null,"id":522190947782643712,"source":"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client<\/a>","in_reply_to_user_id_str":null,"favorited":false,"in_reply_to_status_id":null,"retweet_count":695,"created_at":"Wed Oct 15 01:03:18 +0000 2014","in_reply_to_user_id":null,"favorite_count":226,"id_str":"522190947782643712","place":null,"user":{"location":"Twitter HQ","default_profile":false,"profile_background_tile":true,"statuses_count":40,"lang":"en","profile_link_color":"009999","id":1137751093,"following":false,"protected":false,"favourites_count":0,"profile_text_color":"333333","description":"The Product Security Team at Twitter.","verified":true,"contributors_enabled":false,"profile_sidebar_border_color":"EEEEEE","name":"Twitter Security","profile_background_color":"131516","created_at":"Thu Jan 31 19:37:42 +0000 2013","is_translation_enabled":false,"default_profile_image":false,"followers_count":13852,"profile_image_url_https":"https://pbs.twimg.com/profile_images/3321149170/6bf0ef1ae272b203acfdee4d7e61df49_normal.png","geo_enabled":false,"profile_background_image_url":"http://abs.twimg.com/images/themes/theme14/bg.gif","profile_background_image_url_https":"https://abs.twimg.com/images/themes/theme14/bg.gif","follow_request_sent":false,"entities":{"description":{"urls":[]},"url":{"urls":[{"expanded_url":"https://twitter.com/about/security","indices":[0,23],"display_url":"twitter.com/about/security","url":"https://t.co/E9FCUUoXKF"}]}},"url":"https://t.co/E9FCUUoXKF","utc_offset":-25200,"time_zone":"Pacific Time (US & Canada)","notifications":false,"profile_use_background_image":true,"friends_count":0,"profile_sidebar_fill_color":"EFEFEF","screen_name":"twittersecurity","id_str":"1137751093","profile_image_url":"http://pbs.twimg.com/profile_images/3321149170/6bf0ef1ae272b203acfdee4d7e61df49_normal.png","listed_count":188,"is_translator":false},"coordinates":null},"contributors":null,"text":"RT @twittersecurity: We have disabled SSLv3 protocol support in response to the vulnerability published today. You may need to update your \u2026","geo":null,"retweeted":false,"in_reply_to_screen_name":null,"truncated":false,"lang":"en","entities":{"symbols":[],"urls":[],"hashtags":[],"user_mentions":[{"id":1137751093,"name":"Twitter Security","indices":[3,19],"screen_name":"twittersecurity","id_str":"1137751093"}]},"in_reply_to_status_id_str":null,"id":522339641815748608,"source":"<a href=\"http://itunes.apple.com/us/app/twitter/id409789998?mt=12\" rel=\"nofollow\">Twitter for Mac<\/a>","in_reply_to_user_id_str":null,"favorited":false,"in_reply_to_status_id":null,"retweet_count":695,"created_at":"Wed Oct 15 10:54:09 +0000 2014","in_reply_to_user_id":null,"favorite_count":0,"id_str":"522339641815748608","place":null,"user":{"location":"San Francisco, CA","default_profile":false,"profile_background_tile":true,"statuses_count":3520,"lang":"en","profile_link_color":"0084B4","profile_banner_url":"https://pbs.twimg.com/profile_banners/6253282/1347394302","id":6253282,"following":false,"protected":false,"favourites_count":26,"profile_text_color":"333333","description":"The Real Twitter API. I tweet about API changes, service issues and happily answer questions about Twitter and our API. Don't get an answer? It's on my website.","verified":true,"contributors_enabled":false,"profile_sidebar_border_color":"C0DEED","name":"Twitter API","profile_background_color":"C0DEED","created_at":"Wed May 23 06:01:13 +0000 2007","is_translation_enabled":false,"default_profile_image":false,"followers_count":2418318,"profile_image_url_https":"https://pbs.twimg.com/profile_images/2284174872/7df3h38zabcvjylnyfe3_normal.png","geo_enabled":true,"profile_background_image_url":"http://pbs.twimg.com/profile_background_images/656927849/miyt9dpjz77sc0w3d4vj.png","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/656927849/miyt9dpjz77sc0w3d4vj.png","follow_request_sent":false,"entities":{"description":{"urls":[]},"url":{"urls":[{"expanded_url":"http://dev.twitter.com","indices":[0,22],"display_url":"dev.twitter.com","url":"http://t.co/78pYTvWfJd"}]}},"url":"http://t.co/78pYTvWfJd","utc_offset":-25200,"time_zone":"Pacific Time (US & Canada)","notifications":false,"profile_use_background_image":true,"friends_count":48,"profile_sidebar_fill_color":"DDEEF6","screen_name":"twitterapi","id_str":"6253282","profile_image_url":"http://pbs.twimg.com/profile_images/2284174872/7df3h38zabcvjylnyfe3_normal.png","listed_count":12846,"is_translator":false},"coordinates":null},{"contributors":null,"text":"The OAuth / xAuth issue affecting REST endpoints has been resolved. Please visit our OAuth forums for support https://t.co/XiKCrlfCNX","geo":null,"retweeted":false,"in_reply_to_screen_name":"twitterapi","possibly_sensitive":false,"truncated":false,"lang":"en","entities":{"symbols":[],"urls":[{"expanded_url":"https://twittercommunity.com/category/oauth","indices":[110,133],"display_url":"twittercommunity.com/category/oauth","url":"https://t.co/XiKCrlfCNX"}],"hashtags":[],"user_mentions":[]},"in_reply_to_status_id_str":"515255192787230720","id":515288036242763776,"source":"<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client<\/a>","in_reply_to_user_id_str":"6253282","favorited":false,"in_reply_to_status_id":515255192787230720,"retweet_count":79,"created_at":"Thu Sep 25 23:53:36 +0000 2014","in_reply_to_user_id":6253282,"favorite_count":55,"id_str":"515288036242763776","place":null,"user":{"location":"San Francisco, CA","default_profile":false,"profile_background_tile":true,"statuses_count":3520,"lang":"en","profile_link_color":"0084B4","profile_banner_url":"https://pbs.twimg.com/profile_banners/6253282/1347394302","id":6253282,"following":false,"protected":false,"favourites_count":26,"profile_text_color":"333333","description":"The Real Twitter API. I tweet about API changes, service issues and happily answer questions about Twitter and our API. Don't get an answer? It's on my website.","verified":true,"contributors_enabled":false,"profile_sidebar_border_color":"C0DEED","name":"Twitter API","profile_background_color":"C0DEED","created_at":"Wed May 23 06:01:13 +0000 2007","is_translation_enabled":false,"default_profile_image":false,"followers_count":2418318,"profile_image_url_https":"https://pbs.twimg.com/profile_images/2284174872/7df3h38zabcvjylnyfe3_normal.png","geo_enabled":true,"profile_background_image_url":"http://pbs.twimg.com/profile_background_images/656927849/miyt9dpjz77sc0w3d4vj.png","profile_background_image_url_https":"https://pbs.twimg.com/profile_background_images/656927849/miyt9dpjz77sc0w3d4vj.png","follow_request_sent":false,"entities":{"description":{"urls":[]},"url":{"urls":[{"expanded_url":"http://dev.twitter.com","indices":[0,22],"display_url":"dev.twitter.com","url":"http://t.co/78pYTvWfJd"}]}},"url":"http://t.co/78pYTvWfJd","utc_offset":-25200,"time_zone":"Pacific Time (US & Canada)","notifications":false,"profile_use_background_image":true,"friends_count":48,"profile_sidebar_fill_color":"DDEEF6","screen_name":"twitterapi","id_str":"6253282","profile_image_url":"http://pbs.twimg.com/profile_images/2284174872/7df3h38zabcvjylnyfe3_normal.png","listed_count":12846,"is_translator":false},"coordinates":null}]
```
## Importa el proyecto completo en Eclipse desde este repositiorio
El eclipse que usamos dispone de un plugin de Git. Para importar ese proyecto (así como otros que veremos en clase) sigue este manual:
[Clonar un proyecto de eclipse contenido en un repositorio de Gitlab] (https://gitlab.pervasive.it.uc3m.es/tutorials/clone_project_with_Eclipse_egit)
\ No newline at end of file
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment