JulianHi

I´m an International Business student from Germany, interested in Data Analytics and Machine Learning with a focus on Marketing Applications. My favorite language is R.

You may also like...

149 Responses

  1. Jessica Pitzl says:

    Hi there!
    I can’t get the authentification working:

    Error in get(name, envir = asNamespace(pkg), inherits = FALSE) :
    object ‘.setDummyField’ not found

    Do u know what the problem could be?

    Thanks a lot!
    Jessica

    • julianhi says:

      Hey Jessica!
      Nice to see you on my blog!
      Did you try to reset your workspace and start with a clean one?
      And did you load all the newest versions of the required packages?
      I would like to get some more info about your problem :) Maybe it would help if you post your R code. Maybe we can find a mistake

  2. Jessica Pitzl says:

    oops… wrong copy/paste ;-)

    cred> and then the error appears

    • julianhi says:

      Ok I hope I get it now:
      You define the variables in the beginning and then type:

      twitCred <- OAuthFactory$new(consumerKey=consumerKey,consumerSecret=consumerSecret,requestURL=reqURL,accessURL=accessURL,authURL=authURL)

      And then this error appears?
      Then you should update your R to the newest version. Maybe the twitteR package doesn't work very well with your R version

  3. Jessica Pitzl says:

    oh boy.. why isn’t this showing my code?

    after cred=

    OAuthFactory$new(consumerKey=consumerKey,consumerSecret=consumerSecret,requestURL=requestURL,accessURL=accessURL,authURL=authURL)

    then error…

  4. Jessica Pitzl says:

    the update to 3.0.1 and authorization finally worked
    when i searched Twitter the following happened:

    [1] “SSL certificate problem, verify that the CA cert is OK. Details:nerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed”
    Error in twInterfaceObj$doAPICall(cmd, params, “GET”, …) :
    Error: SSL certificate problem, verify that the CA cert is OK. Details:
    error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

    I did solve it via:

    library(RCurl)
    options(RCurlOptions = list(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”)))

    And now i get my tweets just fine :-)
    Thanks for ur help!

    • julianhi says:

      Oh thanks for the tip! I’ll put it in the blog post. But I guess u’re using windows or? That’s where normally these cert problems appear ;)

      • Jessica Pitzl says:

        jap, i’m using windows ;-)
        I just restarted R. Am I supposed to go through the authorization process everytime I restart?

  5. Jessica Pitzl says:

    Another thing I couldn’t fix so far:

    Warning message: In doRppAPICall(“search/tweets”, n, params = params, retryOnRateLimit = retryOnRateLimit, : 100 tweets were requested but the API can only return 26

    I tried to get tweets from a certain time periode and it wouldn’t give me any results back. Then I tried without a time frame and even then R only returned a few.

    Have u come across that?

    • julianhi says:

      Yeah this is a problem with the new Twitter API. You just got 26 Tweets because Twitter only provided 26. They set a time period and just within this period you can receive the tweets. So now chance till now to get 1 year old tweets or so with the search twitter method

  6. Deepak says:

    Hello there,
    Am using R v3.0.1 and I get this error
    Error in function (type, msg, asError = TRUE) :
    SSL certificate problem, verify that the CA cert is OK. Details:
    error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
    What could I do??

  7. Deepak says:

    Yes am using Windows. . .
    I got the PIN and entered now in R.
    Then when I searchTwitter i get the error
    >>This is my code

    tweets = searchTwitter(“#analytics”, n=1000)
    [1] “SSL certificate problem, verify that the CA cert is OK. Details:nerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed”
    Error in twInterfaceObj$doAPICall(cmd, params, “GET”, …) :
    Error: SSL certificate problem, verify that the CA cert is OK. Details:
    error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

  8. Deepak says:

    Jessica in the previous post said that she fixed using :

    library(RCurl)
    options(RCurlOptions = list(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”)))

    But i get the error as:

    >> the code:

    > library(RCurl)
    > options(RCurlOptions = list(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”)))
    Error: unexpected input in “options(RCurlOptions = list(cainfo = system.file(“”

  9. Deepak says:

    I tried as you said but I get the following error:

    Error: unexpected input in “tweets = searchTwitter(“”

  10. Deepak says:

    Sorry am new to “R”
    Now I just typed out everything and I get:
    >>

    > tweets = searchTwitter(“#analytics”,n=1000,cainfo=”cacert.pem”)
    [1] “Unauthorized”
    Error in twInterfaceObj$doAPICall(cmd, params, “GET”, …) :
    Error: Unauthorized
    >

    • julianhi says:

      Ok now we got a step further;) It says that you are unauthorized to use the GET method of twitter. So there was something wrong in the authentification process. Please type it in line by line. The whole authentification process has to be done every single time you start R. So restart R, go through the auth process and then you can use searchTwitter

  11. Deepak says:

    Thank you so mush. . I have been working on this for a whole week and got into this blog.
    It’s really helpful man.
    If i get into some problems, I’ll ask you. . .

  12. Deepak says:

    Man. . Am tired of this. .I restarted everthing and typed all the code, yet Again I get this error. . .

    library(twitteR)
    > library(ROAuth)
    > reqURL accessURL authURL consumerKey consumerSecret twitCred twitCred$handshake()
    Error in function (type, msg, asError = TRUE) :
    SSL certificate problem, verify that the CA cert is OK. Details:
    error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
    >

  13. Deepak says:

    I Got it man. . .
    I Loaded RCurl and typed a query . it is solved

  14. Deepak says:

    I need help. . We gone a step further man. .
    When am trying to analyze I get this. . . .

    >analysis = score.sentiment(tweets.text, pos.words, neg.words)
    Error in match(words, exc.words) :
    argument “exc.words” is missing, with no default

  15. Deepak says:

    Even if type it as just pos and neg. . .I get the same Error

  16. Deepak says:

    K man. .Here is all my codes. .

    library(twitteR)
    > library(ROAuth)
    > reqURL accessURL authURL consumerKey consumerSecret twitCred twitCred$handshake()
    > library(RCurl)
    > options(RCurlOptions = list(cainfo = system.file(“CurlSSL”,”cacert.pem”,package=”RCurl”)))
    > twitCred$handshake()
    To enable the connection, please direct your web browser to:
    http://api.twitter.com/oauth/authorize?oauth_token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    When complete, record the PIN given to you and provide it here: xxxxxxx
    > local({pkg library(plyr)
    > tweets = searchTwitter(“#Analytics”,n=1000,cainfo=”cacert.pem”)
    > length(tweets)
    [1] 1000
    > tweets.text = laply(tweets, function(t)t$getText())
    > pos = scan(‘positive-words.txt’, what=’character’, comment.char=’;’)
    Read 2006 items
    > neg = scan(‘negative-words.txt’, what=’character’, comment.char=’;’)
    Read 4783 items
    > source(‘sentiment.r’)
    > analysis = score.sentiment(tweets.text, pos.words, neg.words)
    Loading required package: stringr
    Error in FUN(X[[1L]], …) : could not find function “str_split”
    In addition: Warning message:
    In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE, :
    there is no package called ‘stringr’
    > utils:::menuInstallPkgs()
    — Please select a CRAN mirror for use in this session —
    trying URL ‘http://cran.rstudio.com/bin/windows/contrib/3.0/stringr_0.6.2.zip’
    Content type ‘application/zip’ length 75189 bytes (73 Kb)
    opened URL
    downloaded 73 Kb

    package ‘stringr’ successfully unpacked and MD5 sums checked

    The downloaded binary packages are in
    C:UsersTwisterAppDataLocalTempRtmpmCFsdWdownloaded_packages
    > library(stringr)
    > analysis = score.sentiment(tweets.text, pos.words, neg.words)
    Error in match(words, exc.words) :
    argument “exc.words” is missing, with no default
    > q()
    > analysis = score.sentiment(tweets.text, pos.words, neg.words)
    Error in match(words, exc.words) :
    argument “exc.words” is missing, with no default
    > save.image(“C:\Users\Twister\Desktop\Project things\TwitteR\Inital codings”)
    > analysis = score.sentiment(tweets.text, pos.words, neg.words)
    Error in match(words, exc.words) :
    argument “exc.words” is missing, with no default

  17. Deepak says:

    Actually I think there would be some problem in the algorithm because I loaded from another R file. . .I think so. . .

  18. Deepak says:

    Finally it worked. .Thanks a lot to you. . . .Can I get the tweets into an excel. . . .

  19. Deepak says:

    Man,
    Can I get the Positive, Negative tweets in to some kinda Document format. .so that i could know what is negatively spoken . . . .

    • julianhi says:

      The negative tweets are stored in neg.matches in the analysis function. This variable you can write in a document or just display them in R

      • Deepak says:

        Thanks man,
        What is the type of algorithm used?? Is it SVM, NB etc..
        Whats the accuracy level for this analysis??
        If there is a tweet as “This product is not good”, then it will be considered as positive sentiment. . .So what to do for it??

        • julianhi says:

          There is no real algorithm in this example because it´s a very basic one. You just get the tweets and match the containing words with a list of positive and negative words.
          But i´m thinking about writing another sentiment analysis post which uses the viralheat sentiment API as it is fully compatible with R (https://app.viralheat.com/developer/sentiment_api).
          This should deliver more accurate results than the easy wordlist method.

      • Deepak says:

        Can’t get the negative tweets man. . .It says

        > neg.matches
        Error: object ‘neg.matches’ not found

  20. RK says:

    Hi.

    I am getting this error when I am trying to execute the steps mentioned above. Could you please tell me if I am missing something. I am really new to using R

    Error in registerTwitterOAuth(twitCred) :
    oauth has not completed its handshake

    • julianhi says:

      Hi and welcome on my blog. Could you please post the whole output of your R instance? This helps me a lot to answer your question and find the mistake ;)

  21. Tommy says:

    Hi,

    Can i view the tweets?? If so, how. . . .??. . . . .

    • julianhi says:

      Hi,
      in this example you don´t get any tweets from twitter. You just connect to twitter.
      If you want to search twitter you have to use:
      tweets = searchTwitter(“keyword_you_search_for”, n=200, cainfo=”cacert.pem”)

      After that you can just write “tweets” in your R console and all the downloaded tweets are displayed

  22. Ganesan says:

    Hi Julian, I am getting a strange issue with twitteR …

    When I try to do the handshake, the prompt to get the PIN is not waiting enough to get the pin … after couple of seconds, it says “Error: Unauthorized”. I have pasted the session info in the end.

    credential <- OAuthFactory$new(consumerKey="key",consumerSecret="key",requestURL="https://api.twitter.com/oauth/request_token&quot; ,accessURL="https://api.twitter.com/oauth/access_token&quot;,authURL="https://api.twitter.com/oauth/authorize&quot;);

    credential$handshake(cainfo="cacert.pem")

    To enable the connection, please direct your web browser to:
    https://api.twitter.com/oauth/authorize?oauth_token=a8fLZZmWR2NnAtnL9uYlVZf11ntaKrO1SvNJmrzkPwg
    When complete, record the PIN given to you and provide it here:
    Error: Unauthorized

    Session Info:
    R version 3.0.1 (2013-05-16)
    Platform: x86_64-w64-mingw32/x64 (64-bit)

    locale:
    [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252
    [3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C
    [5] LC_TIME=English_United Kingdom.1252

    attached base packages:
    [1] stats graphics grDevices utils datasets methods base

    other attached packages:
    [1] twitteR_1.1.7 rjson_0.2.12 ROAuth_0.9.3 digest_0.6.3 RJSONIO_1.0-3 RCurl_1.95-4.1
    [7] bitops_1.0-5

    loaded via a namespace (and not attached):
    [1] tools_3.0.1

  23. Deepak says:

    Hey there. . . .Do you have any tutorials regarding ARIMA in R. . .And is it worth to predict the future values using ARIMA??? Am actually doing a project regarding it. . .

  24. Tonie says:

    What kind of sampling is this???

  25. Deepak raj says:

    When we use the command:
    >SearchTwitter(“#Keyword”,n=1500)
    So, n=1500 is what kind of sampling method.
    And the second query is can we use combination of words like “#HCL salary” like wise. .So how can I do that…

    • julianhi says:

      Hey Deepak raj
      SearchTwitter gets real-time tweets. So you get the 1500 newest tweets matching your search term. In an upcoming release Twitter will change this to popular tweets. So the API will just release popular tweets by default. But they will publish some information before the release.

      TwitteR is just a parser to translate the R code and call the Twitter API with your parameters. So you just have the option to use one search string. But this search string can contain everything you want. It can contain accounts like “@julianhi” and also hashtags. So your example “#HCL salary” gives you tweets containing the hashtag #HCL and the word salary.
      Hope i could help you

      • Deepak says:

        Yes. I have tried a lot of times using “#HCL salary” but the API returns not even a single tweet :-(

        • julianhi says:

          Hey!
          The problem in your case is the keyword.
          If you go to the normal twitter website and search for “#hcl salary” there are just something about 5 tweets ever written with these keywords. And the twitteR search which uses the Twitter API can just return recent ones, but the last tweet containing your keywords was written months ago.
          So in other words: there are just no recent tweets containing your keywords

  26. Hello !
    Thanks a lot for this tutorial.
    Simple question : is there any way to move (or backup) the cacert.pem file to use it to avoid doing the handshake each time ?
    For example, I’m actually doing the handshake in a the R terminal, because there is a bug in Rstudio about copy/paste the url (mentionned for example here : http://stackoverflow.com/questions/15958512/invalid-request-token-using-oauth-and-r-is-the-r-code-correct). Is it possible to use in R Studio the cacert.pem file handshaked by the terminal ?

    • julianhi says:

      Hey Jonathan!
      The handshaked cacert file should be saved on your system. So you just be able to use it with RStudio.
      I also had this problems with RStudio and i´ll take a look at it again. I´ll send you some feedback within the next days

  27. Deepak says:

    Hi there, Another query:

    I tried something new and and I get lot of errors. Plz help me out in this.

    1)got the tweets about specific keyword and I also copied some of the reviews in websites.
    How could I append it to the tweets and do the same sentiment analysis.

    2)What is the code to copy all the tweets to a file.

    3)can I get the tweets in a file and add those reviews from websites along with it and read the file again in R and do the same sentiment analysis. If Yes, tell me how plz…

  28. Jani says:

    Julianhi,
    at first I like to than you for your website. It has been very helpful. My question concerns retrieving tweets. Is it possible to retrieve tweets that were created, let’s say, in the past two years? Not every tweet generated but for a certain keyword, e.g., “Samsung”.

    • julianhi says:

      Hey Jani,
      Thanks for your comment.
      But it’s not possible to get tweets older than let’s say 3 months via the Twitter API. That’s caused by Twitter’s restrictions. You can just get these Tweets from an official Twitter Data Reseller with full firehose access like GNIP. But these services are very expensive.
      I hope I could help you.

  29. clark says:

    hi,
    i’m a fresher with R. thanks for your guidance
    i got a ‘ true’ after complete your code , but I cannot search tweets…

    “SSL certificate problem, verify that the CA cert is OK. Details:nerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed”
    Error in twInterfaceObj$doAPICall(cmd, params, “GET”, …) :
    Error: SSL certificate problem, verify that the CA cert is OK. Details:
    error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

    wats the problem?

    • julianhi says:

      Hey clark,
      do you have some more information for me?
      Do you use windows and RStudio?

      • Claudia says:

        Hi julianhi
        compliments for your work and blog. I’m italian so I do not speak English very well, so sorry in advance. I have a problem to complete the authentication phase. Oauth has not completed its handshake because answer: Errore Forbidden. This is my script:
        > require(twitteR)
        >
        > reqURL accessURL authURL consumerKey consumerSecret if (!file.exists(‘cacert.perm’)){
        + download.file(url = ‘http://curl.haxx.se/ca/cacert.pem’,
        + destfile=’cacert.perm’)
        + }
        provo con l’URL ‘http://curl.haxx.se/ca/cacert.pem’
        Content type ‘unknown’ length 241697 bytes (236 Kb)
        URL aperto
        downloaded 236 Kb

        > twitCred curl = getCurlHandle()
        > options(RCurlOptions = list(capath = system.file(‘CurlSSL’, ‘cacert.pem’,
        + package = ‘RCurl’), ssl.verifypeer = FALSE))
        > curlSetOpt(.opts = list(proxy = ‘proxyserver:port’), curl = curl)
        An object of class “CURLHandle”
        Slot “ref”:

        > twitCred$handshake()
        To enable the connection, please direct your web browser to:
        http://api.twitter.com/oauth/authorize?oauth_token=vWwCyCh1sZ8iQyaQEl2nFcV1FUR1JsOrAt0EpXfycUs
        When complete, record the PIN given to you and provide it here: xxxxxxxxxxxxxx
        Errore: Forbidden
        > #verifica
        > registerTwitterOAuth(twitCred)
        Errore in registerTwitterOAuth(twitCred) :
        oauth has not completed its handshake
        >

        I have searched the internet for a solution but nothing. I’m in the stall and I can not go on. You can help me?

        • julianhi says:

          Hey Claudia,
          Thanks for reaching out and sorry for the late answer. I’ve updated the code and the tutorial and the new code should solve your problem. Let me know if it worked.
          Regards

  30. Deepak says:

    Hey there,
    Can I get tweets on some past date???
    If I want to take tweets on 1st November 2013, Can I do it?

  31. R Jagadeesh says:

    I am getting error msg after this line… can you please help me find a solution?
    download.file(url= “http://curl.haxx.se/ca/cacert.pem”, destfile = “cacert.pem”)
    Error in download.file(url = “http://curl.haxx.se/ca/cacert.pem”, destfile = “cacert.pem”) :
    cannot open destfile ‘cacert.pem’, reason ‘Permission denied’

  32. satya says:

    i want to code for twitter follower and friend list using R code (output will be get in the form of matrix) .give me guidiance

  33. satya says:

    is it any possible for other(public page) facebook a/c using R code

  34. mkminn says:

    Hi Julian – I received less errors using this piece of good, but I am still having trouble. You have been really helpful with some of the others on your blog and thought you might spare a minute or two. I can get my PIN but I’m still “Forbidden” – ouch!

    ## Not run:
    > ## A real example, but using a fictitious consumerkey and consumer
    > ## secret – you’ll need to supply your own
    > reqURL accessURL authURL consumerKey consumerSecret twitCred twitCred$handshake()
    To enable the connection, please direct your web browser to:
    http://api.twitter.com/oauth/authorize?oauth_token=Hzn2Sk7ljRTA2rOdBXhJAe0JQscNlY5wCIf6OBkT7M
    When complete, record the PIN given to you and provide it here:

    registerTwitterOAuth(twitCred)
    Error: Forbidden

  35. vivek ratan says:

    Hi Julianhi,
    i used the same above code with my consumer key & secert key. Initailly this code was working on my PC but now i am facing an error :Forrbiden.

    reqURL <- "http://api.twitter.com/oauth/request_token&quot;

    accessURL <- "http://api.twitter.com/oauth/access_token&quot;

    authURL <- "http://api.twitter.com/oauth/authorize&quot;

    consKey <- "Myconsumerkey"

    consSecret <- "Mysecrectkey"

    twitCred twitCred$handshake(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”))
    Error: Forbidden

    Please help me. to resolve it. i need urgently to complete my POC.
    contact me:- vratan009@gmail.com
    vivek

    • julianhi says:

      Hey Vivek,
      I updated my code just a few days ago, because Twitter changed the API access options.

      But if you use the code above everything should work find. I see you still used the old code.
      Regards

  36. vivek ratan says:

    Hey julianhi,

    Thanks, i ran the above code and it ran fine. Thanks for providing the quick response.

    Regards

  37. CJ says:

    Hi Julian, first of all, thanks so much for this blog and your tutorials, they have been incredibly helpful.

    I followed your instructions using R instead of RStudio since I couldn’t copy the URL there. Everything works fine until I get to the final line of the code. Then I get the error: “oauth has not completed its handshake”.

    The file cacert.pem is in the folder I set as the working directory. I checked the consumer key and secret and updated R and all packages to the latest version.

    I’m really not sure what went wrong. Have you come across this?

    Your help would be very much appreciated. Thanks!

  38. CJ says:

    Update: After a bit of ‘trial and error’, I found the solution: In the application settings, I changed the Application Type to “Read, Write and Access direct messages”. Then I had to log out and log in again, and now it works. Just in case anyone else has the same problem. :)

  39. FX Jollois says:

    Hi,

    I try to use the presented code, and I can’t complete the handshake. Here is the executed code and the error messages given by R.

    thanks for help, if possible.
    Regards
    FX Jollois

    > # Set SSL certs globally
    > options(RCurlOptions = list(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”)))
    > # Request token URL https://api.twitter.com/oauth/request_token
    > reqURL # Authorize URL https://api.twitter.com/oauth/authorize
    > authURL # Access token URL https://api.twitter.com/oauth/access_token
    > accessURL # Consumer key
    > consumerKey # Consumer secret
    > consumerSecret twitCred twitCred$handshake(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”))
    To enable the connection, please direct your web browser to:
    http://api.twitter.com/oauth/authorize?oauth_token=
    When complete, record the PIN given to you and provide it here: …
    Error: Forbidden
    > registerTwitterOAuth(twitCred)
    Error in registerTwitterOAuth(twitCred) :
    oauth has not completed its handshake

    • julianhi says:

      Hey FX Jollois,
      Did you add your consumer key and access token you received when you created your app on twitter?
      Regards

    • julianhi says:

      An other user had a similar problem. Maybe you try his solution:
      “After a bit of ‘trial and error’, I found the solution: In the application settings, I changed the Application Type to “Read, Write and Access direct messages”. Then I had to log out and log in again, and now it works. “

  40. FX Jollois says:

    Yes ! I have just remove them to publish the code. And I just see that the message is quietly diffrent from how I write it.

    Regards
    FX Jollois

  41. FX Jollois says:

    Nope… Still don’t work… I will try later. Maybe patience is the better way…

    Regards
    FX Jollois

  42. FX Jollois says:

    It is fixed ! I add “s” to “http” and everything works ! I have missed my mistake .

    Thanks for help, and many thanks for the code !
    FX

  43. Claudia says:

    Hi Julianhi
    I solved it so:
    #load data authentication previously saved
    load(“twitter_authentication.Rdata”)
    library(RCurl)
    curl = getCurlHandle()
    options(RCurlOptions = list(capath = system.file(‘CurlSSL’, ‘cacert.pem’,package = ‘RCurl’), ssl.verifypeer = FALSE))
    curlSetOpt(.opts = list(proxy = ‘proxyserver:port’), curl = curl)
    twitCred$handshake()
    registerTwitterOAuth(twitCred)

    Regards

  44. Alessandro says:

    Hi Julian,

    Clear explanation, as always. I have one problem: when I try to connect, just after I am asked to enter my PIN (but before I have any chance to do so), I receive an “Error: Unauthorized”. I cannot make my head around what could be causing this behaviour. I tested it from different machines and with/without proxy, nothing worked so far.

    I also open a question on Stack Exchange, where I posted the whole code: http://stackoverflow.com/questions/21682266/unauthorized-error-with-twitter-oauth

    • julianhi says:

      Hey Alessandro,
      this problem often occurs when you enter the whole code at once and hit enter.
      You should run the code line by line. This should solve your problem.
      Please let me know if this worked for you.

      Regards

  45. Nikhil Tuli says:

    Hi Julian,

    I am following certain users on twitter as part of my research project.

    I seek to download tweets through R software for the following three scenarios (Assuming user as XYZ)

    1. What XYZ is tweeting?

    2. What people are saying about XYZ on twitter?

    3. What are the tweets made by XYZ that are being retweeted by users?

    For the first scenario, I am make use of the following code:
    rdmTweets.list <- userTimeline('@xyz', n=100, cainfo="cacert.pem")
    Is this the right approach?

    I am making use of following code to download mentions of XYZ on twitter. But the first line also takes into account retweets.
    original.list<-searchTwitter('@xyz',n=1000, cainfo="cacert.pem")
    original.list<-searchTwitter('#xyz',n=1000, cainfo="cacert.pem")

    Kindly comment as to how I can separately download tweets for the above mentioned three scenarios.

    TIA.

    Regards,
    Nikhil.

    • julianhi says:

      Hey Nikhil,
      sorry for the late answer.

      1.) Yes, this is the right approach to get the users tweets.

      2.) You can find the retweets with the following code. It helps you to find the retweets and then you can delete them from your list.

      # regular expressions to find retweets
      grep(“(RT|via)((?:\b\W*@\w+)+)”, tweets,
      ignore.case=TRUE, value=TRUE)

      # which tweets are retweets
      rt_patterns = grep(“(RT|via)((?:\b\W*@\w+)+)”,
      tweet_txt, ignore.case=TRUE)

      # show retweets (these are the ones we want to delete)
      tweet_txt[rt_patterns]

      I used it in this tutorial:
      http://thinktostart.wordpress.com/2013/06/25/visualize-retweets-with-r/

      3.) When you download the user timeline, you can find the retweet value in the answer. It is an answer
      which tells you how often it was retweeted.
      To find these retweets could be a little bit more difficult.

      I hope I could help you.
      If you have any further questions feel free to ask.

      Regards

  46. Nikhil Tuli says:

    Hi Julian,

    I have downloaded twitter data using R in .csv format. These are the fields :

    text favorited favoriteCount replyToSN created truncated replyToSID id replyToUID statusSource screenName retweetCount isRetweet retweeted longitude latitude

    For a given tweet in .csv file, the value of retweetCount>0 though the value for retweeted=0. I am not able to exactly discern the meaning for these three different fields (retweetCount, isRetweet, retweeted).

    Could you please help me understand the same?

    TIA

    • julianhi says:

      Hey Nikhil,
      So
      retweetCount – How often it was retweeted
      isRetweet – Is this tweet a retweet or an original tweet?
      retweeted – Was the tweet retweeted? (true/false)

      So there seems to be a problem with the API. I don´t really know why it returns these values but I found a lot of similar questions in the Twitter forum but no real answer.
      I will take a look at this topic again.

      Regards

  47. Nikhil Tuli says:

    A small clarification with respect to the above post

    Value for retweeted is FALSE even though retweetCount>0.

  48. Onkar khaladkar says:

    Hi julian
    I am calculating sentiment from twitter tweets.I have a question can we fetch conversations from a single tweet that is fetched in search twitter function.

    • julianhi says:

      Hey Onkar,
      I think that this is not possible on the direct way. You can just search all the tweets you downloaded for connections. To have access to the whole Twitter data you would need something like a Firehose access.

      Regards

  49. phdcs says:

    I get the following error by executing : twitCred$handshake(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”)):
    Error in function (type, msg, asError = TRUE) :
    Could not resolve host: api.twitter.com; Host not found, try again

    how to resolve this issue?

  50. Tasos says:

    Well after I make the authentication and all the necessary, I get this as a result!

    twitterMap(“socialmedia2day”, fileName=”Map_socialmedia2day.pdf”, nMax=3000)
    Getting data from Twitter, this may take a moment.
    Error in twInterfaceObj$doAPICall(paste(“users”, “show”, sep = “/”), params = params, :
    OAuth authentication is required with Twitter’s API v1.1

    Problem on twitter????

    • julianhi says:

      Hey Tasos,
      Did you receive a “true” after the authentication process?

      Regards

    • julianhi says:

      Hey Tasos,
      it seems like you forgot the last row in the code.
      The authentication is not done after you inserted the PIN. You have to execute the line:
      registerTwitterOAuth(twitCred)
      This line will then, if everything works well, return a “TRUE”.
      Then you are ready to go.

      Regards

  51. Barns says:

    Hey Julian!

    Everything works until the point:

    > twitCred$handshake(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”))
    Error: Unauthorized
    >
    > registerTwitterOAuth(twitCred)
    Error in registerTwitterOAuth(twitCred) :
    oauth has not completed its handshake

    Any idea?

    • julianhi says:

      Hey Barns,
      You have to execute the code line by line. Did you do so? Because otherwise the code skips the line where you have to enter your PIN.

      Regards

      • Barns says:

        Yep, did it line by line, but after running line 14, console says “Error: Unauthorized” :/ The keys are definetely correct (Used Mozilla and my personal Account)

        • julianhi says:

          Could you please copy-and-paste your R history here? It could help me finding the error.

          Regards

          • Barns says:

            Thanks again for your quick response! Somehow i got it yesterday until that point. I think the problem was a double-twitter-login with two different accounts…

      • mrplume says:

        same problem, but I use just one twitter account

        R logs

        > library(twitteR,igraph)
        Le chargement a nécessité le package : ROAuth
        Le chargement a nécessité le package : RCurl
        Le chargement a nécessité le package : bitops
        Le chargement a nécessité le package : digest
        Le chargement a nécessité le package : rjson
        > library(RCurl)
        > options(RCurlOptions = list(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”)))
        > require(twitteR)
        > reqURL accessURL authURL apiKey apiSecret twitCred twitCred$handshake(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”))
        Erreur : Unauthorized
        > twitCred$handshake(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”))
        Erreur : Unauthorized

        can you help me please ?

  52. Supri Amir says:

    library(twitteR)
    library(stringr)
    library(RCurl)

    reqURL <-"https://api.twitter.com/oauth/request_token&quot;
    accessURL <- "http://api.twitter.com/oauth/access_token&quot;
    authURL <- "http://api.twitter.com/oauth/authorize&quot;
    consumerKey="xxxxxxxxxxx" % i have this key
    comsumerSecret="xxxxxxxxxx" % i have this key

    download.file(url="http://curl.haxx.se/ca/cacert.pem&quot;,destfile="cacert.pem") % for what??

    twitCred <- OAuthFactory$new(consumerKey=consumerKey,
    consumerSecret=consumerSecret,
    requestURL=reqURL,
    accessURL=accessURL,
    authURL=authURL)

    twitCred$handshake(cainfo="cacert.pem") # curl is a command line tool for transfering
    registerTwitterOAuth(twitCred)

    Hai julian,

    I wanna ask you several question.

    1. I try to run this code but still error because "unauthorized". i am working with mac os.
    How to solve it?

    2.This code for what?? should i download this file?
    download.file(url="http://curl.haxx.se/ca/cacert.pem&quot;,destfile="cacert.pem")

    3.it there have different setting for the cacert.pem???

  53. Stu says:

    Please help!! I am having “unauthorized” when started searching. This problem has been several days. So I decided to create another application today. Could that be my application setting? Below is my R console output:

    > Cred$handshake(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”))
    To enable the connection, please direct your web browser to:
    https://api.twitter.com/oauth/authorize?oauth_token=YZGu3jRN2ZxG9IwvhvEJMcR9oXTw2GcV8geGp49ME
    When complete, record the PIN given to you and provide it here: 8357550
    > registerTwitterOAuth(Cred)
    [1] TRUE
    > save(Cred, file=”twitter authentication rkresnadi1.Rdata”)
    > twitlst <- searchTwitter("sustainable development", n=1000, cainfo="cacert.pem")
    [1] "Unauthorized"
    Error in twInterfaceObj$doAPICall(cmd, params, "GET", …) :
    Error: Unauthorized

    • julianhi says:

      Hey Stu,
      your code seems to be ok. I saw that you changes the name of the Credentials Object to Cred, but that shouldn´t be a problem if you did this in your whole code. But you receive a “TRUE” which normally tells you that the authentication was successful. Could you also please post the first part of your code?
      Regards

  54. nolly says:

    please why am i getting error: forbidden
    > library(twitteR)
    Loading required package: ROAuth
    Loading required package: RCurl
    Loading required package: bitops
    Loading required package: digest
    Loading required package: rjson
    Warning messages:
    1: package ‘twitteR’ was built under R version 3.0.2
    2: package ‘ROAuth’ was built under R version 3.0.2
    3: package ‘RCurl’ was built under R version 3.0.2
    4: package ‘digest’ was built under R version 3.0.2
    > library(plyr)

    Attaching package: ‘plyr’

    The following object is masked from ‘package:twitteR’:

    id

    Warning message:
    package ‘plyr’ was built under R version 3.0.2
    > library(stringr)
    Warning message:
    package ‘stringr’ was built under R version 3.0.2
    > library(ROAuth)
    > download.file(url=”http://curl.haxx.se/ca/cacert.pem”, destfile=”cacert.pem”)
    trying URL ‘http://curl.haxx.se/ca/cacert.pem’
    Content type ” length 250283 bytes (244 Kb)
    opened URL
    downloaded 244 Kb

    > requestURL accessURL = “http://api.twitter.com/oauth/access_token”
    > authURL consumerKey = “myKey”
    > consumerSecret = “mySecret”
    > Cred Cred$handshake(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”) )
    Error: Forbidden

  55. satya says:

    where is updated code in your blog please give me your link

    • julianhi says:

      Hey satya
      The updated code is the code above. But the code used in the question before was an Old version.
      So just use the code from the Tutorial and everything should work.

      Regards

  56. CMG says:

    Hi julianhi,

    I’m having the same problem retrieving tweets.

    > tweets = searchTwitter(“sustainable development”, n=200, cainfo=”cacert.pem”)
    [1] “Unauthorized”
    Error in twInterfaceObj$doAPICall(cmd, params, “GET”, …) :
    Error: Unauthorized

    Here is my code, I’m putting it in line by line. Using RStudio Version 0.98.501 on Mac.

    > library(RCurl)
    > options(RCurlOptions = list(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”)))
    > require(twitteR)
    > reqURL accessURL authURL apiKey apiSecret twitCred twitCred$handshake(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”))
    To enable the connection, please direct your web browser to:
    https://api.twitter.com/oauth/authorize?oauth_token=KiysFMXkw1ggdHGviVv5U8Yef4ooZxw7AZ89L1LSmFM
    When complete, record the PIN given to you and provide it here: 1234567
    > registerTwitterOAuth(twitCred)
    [1] TRUE
    > tweets = searchTwitter(“sustainable development”, n=200, cainfo=”cacert.pem”)
    [1] “Unauthorized”
    Error in twInterfaceObj$doAPICall(cmd, params, “GET”, …) :
    Error: Unauthorized

    Please help!

    CMG

    • CMG says:

      Hi J,

      here’s the complete code, I messed up the last post. Like I said, I’m using RSudio on Mac

      > library(RCurl)
      Loading required package: bitops
      > options(RCurlOptions = list(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”)))
      > require(twitteR)
      Loading required package: twitteR
      Loading required package: ROAuth
      Loading required package: digest
      Loading required package: rjson
      > reqURL accessURL authURL apiKey apiSecret twitCred twitCred$handshake(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”))
      To enable the connection, please direct your web browser to:
      https://api.twitter.com/oauth/authorize?oauth_token=epAv0cZJWd7sRayMvKwDPVjQtPiVnmX1dK1AKEZjVg
      When complete, record the PIN given to you and provide it here: 1234567
      > registerTwitterOAuth(twitCred)
      [1] TRUE
      > tweets = searchTwitter(“MMR”, n=200, cainfo=”cacert.pem”)
      [1] “Unauthorized”
      Error in twInterfaceObj$doAPICall(cmd, params, “GET”, …) :
      Error: Unauthorized

      • CMG says:

        Ok, there isn’t enough room to put up the whole code, can I email it to you?

        • julianhi says:

          Hey CMG,
          sorry for the late answer but I was very busy.
          I can´t find a bug in your code at the moment.
          But did you try do execute it in the normal R console?
          Sometimes the authentication does not really complete in RStudio.

          Regards

  57. Darth_vedR says:

    Hello, I am having also the error forbidden during the handshake:

    To enable the connection, please direct your web browser to:
    http://api.twitter.com/oauth/authorize?oauth_token=FWJevpPdDnZrbuqmmkyrQzcUZzRNR3we83LlJdAXnY
    When complete, record the PIN given to you and provide it here: 1817865
    Error: Forbidden

    And I tried many times…..

    here is my code:

    library(twitteR)
    library(RCurl)
    # Set SSL certs globally
    options(RCurlOptions = list(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”)))

    require(twitteR)
    reqURL <- "https://api.twitter.com/oauth/request_token&quot;
    accessURL <- "http://api.twitter.com/oauth/access_token&quot;
    authURL <- "http://api.twitter.com/oauth/authorize&quot;
    consumerKey <- "xpArd5lrnnjxxxxxxxxxxxxxxxxxx"
    consumerSecret <- "jFF4fG3wrhx7Ya8P8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
    twitCred <- OAuthFactory$new(consumerKey=consumerKey,
    consumerSecret=consumerSecret,
    requestURL=reqURL,
    accessURL=accessURL,
    authURL=authURL)

    twitCred$handshake(cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl"))
    registerTwitterOAuth(twitCred)

    What did I miss ? Do you think SSL is not working properly ?

  58. Frank says:

    Good afternoon, I’ve read through the comments and while I saw the same issue (Error: Unauthorized) I didn’t see a way to resolve it.

    > twitCred$handshake(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”))
    To enable the connection, please direct your web browser to:
    https://api.twitter.com/oauth/authorize?oauth_token=xyz
    When complete, record the PIN given to you and provide it here:

    You mention to use the most updated code but I think I am. Here’s what I’m using:

    library(RCurl)
    # Set SSL certs globally
    options(RCurlOptions = list(cainfo = system.file(“CurlSSL”, “cacert.pem”, package = “RCurl”)))

    require(twitteR)
    reqURL <- "https://api.twitter.com/oauth/request_token&quot;
    accessURL <- "https://api.twitter.com/oauth/access_token&quot;
    authURL <- "https://api.twitter.com/oauth/authorize&quot;
    apiKey <- "removed"
    apiSecret <- "removed"

    twitCred <- OAuthFactory$new(consumerKey=apiKey,consumerSecret=apiSecret,requestURL=reqURL,accessURL=accessURL,authURL=authURL)

    twitCred$handshake(cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl"))

    registerTwitterOAuth(twitCred)

  59. Rodrigo Nifla Cornejo says:

    Hi Julianhi i got this error can u help me please?

    Error in check_twitter_oauth() : OAuth authentication error:
    This most likely means that you have incorrectly called setup_twitter_oauth()’

    • JulianHi says:

      Hey
      could you please post the code until the error appears?

      Regards

      • Rodrigo Nifla Cornejo says:

        this is my code

        install.packages(c(“devtools”, “rjson”, “bit64″, “httr”))

        #RESTART R session!

        library(devtools)
        install_github(“twitteR”, username=”geoffjentry”)
        library(twitteR)

        api_key <- "mwcOI7mNkPUhxxxxxxxxxxxxxxx"

        api_secret <- "BwyN7OvghbPSW9RDSaSzVu0aRhhxxxxxxxxxxxxxxxxxxxxxxx"

        access_token <- "2829195264-D8HgRXZCyUd193lxxxxxxxxxxxxxxx"

        access_token_secret <- "2829195264-D8HgRXZCyUd193lC5ILEFxxxxxxxxxxxxxxx"

        setup_twitter_oauth(api_key,api_secret,access_token,access_token_secret)

        and after setup_twitter_oauth is executed the erros appears

        Error in check_twitter_oauth() : OAuth authentication error:
        This most likely means that you have incorrectly called setup_twitter_oauth()’

        i think httr is 401 but i dont know how to change it

        thanks

  60. i hope you can read my poste.
    I followed the instruction step by step, and I still have this problem: ( Erreur : Authorization Required twitter )

    my code is :

    requestURL<- "https://api.twitter.com/oauth/request_token&quot;
    accessURL <- "https://api.twitter.com/oauth/access_token&quot;
    authURL <- "https://api.twitter.com/oauth/authorize&quot;
    consumerKey <- "..scrt….IX4hvp7lm00"
    consumerSecret <- "REoChpe…scrt……..I0"
    Cred <- OAuthFactory$new(consumerKey = consumerKey,
    consumerSecret = consumerSecret,
    requestURL = requestURL,
    accessURL = accessURL,
    authURL = authURL)
    Cred$handshake()

    can you please help me.

    PS: My OS is opensue 12.3

    • JulianHi says:

      Hey
      where exactly does this error appear?
      I have never seen this error before. Did you load the twitteR package?

      Regards

    • rayo says:

      Hope u dont mind if i comment.
      i ran into this same error but my advice is to follow the same step thats in this blog not this one you are using. You might want to try it on the Rgui first not rstudio.
      if it works on the Gui refresh rstudio n try it the authentication through there.

      if you still want to use this then try checkin you token n keys they might have changed.
      but if you get this error when you to register the oauth then i think registeroauth is no longer a function.

      hop this helps

  61. rayo says:

    Hi when i try to download the twitteR package from your github account i keep getting
    “Warning message:
    Username parameter is deprecated. Please use geoffjentry/twitteR”

  62. nurandi says:

    Thank you, Julian. I find your blog is useful.
    I followed tutorial from you to get twitter authentication a couple days ago, and it worked. But now, after updated R and twitteR package to the latest version, it return error message when execute setup_twitter_oauth() function. “Error: could not find function “setup_twitter_oauth”. twitteR package and dependencies have been loaded properly. Would appreciate any suggestion and advice.

  63. mattia says:

    Hi, I’m trying to use tha package twitteR, following what is written above.
    I installed the package as you said, and then:

    library(twitteR)

    api_key <-"…"

    api_secret <- "…"

    access_token <- "…"

    access_token_secret <- "…"

    setup_twitter_oauth(api_key,api_secret,access_token,access_token_secret)

    But this error occurs:

    [1] "Using direct authentication"
    Errore in check_twitter_oauth() : OAuth authentication error:
    This most likely means that you have incorrectly called setup_twitter_oauth()'

    I don't know what is it!! can you help me?

    thank you !
    Mat

    • JulianHi says:

      Hey
      do you have the most recent version of the twitteR package?
      And are you using RStudio and/or Windows?

      Regards

      • Mattia says:

        Hi! Two days ago I re-install R and twitteR package and now it is working!
        But one more question: I see that I can download tweets of the past, but only for the 7 previous days! I need to use older tweets? Do you have any suggestions (I’m a student, I can’t pay too much for having them).
        Thank you,
        Mattia

        • JulianHi says:

          Hey
          this is because of restrictions of the Twitter API. You can just get recent tweets.
          Twitter does not talk about this limits in detail and they can vary.
          So there is no way to get old tweets. Your only way in the moment is saving the results every day and get all new tweets.

          I hope that Twitter will open their archive they created recently where they allow you to search all tweets ever written.
          Maybe they will make it accessible through their API. I will update the post if I hear something new.

          Regards

    • rayo says:

      @mattia try running it first on the R GUI not rstudio. Ensure u install twitteR package from Github

  1. June 25, 2013

    @mattia try running it first on the R GUI not rstudio. Ensure u install twitteR package from Github

  2. July 8, 2013

    @mattia try running it first on the R GUI not rstudio. Ensure u install twitteR package from Github

  3. July 30, 2013

    @mattia try running it first on the R GUI not rstudio. Ensure u install twitteR package from Github

  4. September 2, 2013

    @mattia try running it first on the R GUI not rstudio. Ensure u install twitteR package from Github

  5. September 9, 2013

    @mattia try running it first on the R GUI not rstudio. Ensure u install twitteR package from Github

  6. March 5, 2014

    @mattia try running it first on the R GUI not rstudio. Ensure u install twitteR package from Github

  7. March 23, 2014

    @mattia try running it first on the R GUI not rstudio. Ensure u install twitteR package from Github

  8. May 20, 2014

    @mattia try running it first on the R GUI not rstudio. Ensure u install twitteR package from Github

  9. June 25, 2014

    @mattia try running it first on the R GUI not rstudio. Ensure u install twitteR package from Github

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">