KGI

blog blog blog

Twitter GitHub LinkedIn Lanyrd Email Feed

Talking to Yourself: A Twitter Bot in Clojure as a Total Newb

Although the ermahgerd API we called in our first explorations in Clojure is pretty much endlessly useful, we want to push our Clojure non-abilities farther with something a bit more complicated–and what could be more complicated than Twitter?

We’ll start by making a restful call with the twitter-api library. After running lein new app twitterbot to set up a project, we’ll need to declare a dependency on twitter-api in our project.clj by adding [twitter-api "0.7.5"] to the :dependencies vector. Next, our Twitter bot won’t be much fun without some Twitter. Start a new account and set yourself up with a new application. Authorize your bot-self for it with read permission; we’ll come back to get write permission later. For a bot, a dev.twitter.com token is sufficient. Go ahead and tweet at the baby bot a few times, so we can retrieve the mentions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
(ns tipbot.core
  (:use
   [twitter.oauth]
   [twitter.callbacks]
   [twitter.callbacks.handlers]
   [twitter.api.restful])
  (:import
   (twitter.callbacks.protocols SyncSingleCallback)))

(def my-creds (make-oauth-creds "my-special-personal-app-consumer-key"
                                "none-of-your-business-app-consumer-secret"
                                "mine-and-only-mine-user-access-token"
                                "no-you-cant-have-my-user-access-token-secret"))

(defn -main
  []
  (println
      (statuses-mentions-timeline :oauth-creds my-creds
                                :params {:screen-name "abotcalledquest"})))

This prints out the entire, wordy HTTP response. The library provides predefined callbacks. Let’s try one that only returns the body of the response:

1
2
3
4
5
6
7
8
(defn -main
  []
  (println
      (statuses-mentions-timeline :oauth-creds my-creds
                                  :params {:screen-name "abotcalledquest"}
                                  :callbacks (SyncSingleCallback. response-return-body
                                                                  response-throw-error
                                                                  exception-rethrow)))

That slims the response down some, but let’s try to get just the text of the tweets. There’s a handy type method that we can use to figure out that we’re dealing with a vector of maps. We learned all kindsa ways to extract a value from a map last time, but how to do that for each value in the vector of tweets? Well, as a proud Lambda Lady I know that the superior way to iterate is a higher-order map function–and indeed, Clojure has a tidy map function. Yes, confusingly, this has nothing to do with the map datastructure. Nor does it have anything to do with cartography, though that’s also awesome. Onward!

EDIT My instincts about map were sort of wrong as it turns out! See this excellent comment from Zane on why we should wrap something that maps over a sequence in dorun.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
(defn -main
  []
  (dorun
      (println 
          (map :text
              (statuses-mentions-timeline :oauth-creds my-creds
                                      :params {:screen-name "abotcalledquest"}
                                      :callbacks (SyncSingleCallback. response-return-body
                                                                      response-throw-error
                                                                      exception-rethrow))))))

                                                                                                          We want the user as well as the text, though, so were going to use [select-keys](http://blog.jayfields.com/2011/01/clojure-select-keys-select-values-and.html).

select-keys takes the response map as the first param though. How can we pass this function to map? In Scala we’d use an underscore as a placeholder.

To #clojure IRC! Alan Malloy showed me that if I preface the function with #, I can use % as a placeholder, as in #(select-keys % [:text :user]), which can be passed to map. This, as it turns out, is shorthand (via a special-case reader macro) for fn, the way to declare an anonymous function in Clojure. #(select-keys % [:text :user]) is the same as writing (fn [x] (select-keys x [:text :user])). Handy!

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
(defn -main
  []
  (dorun 
      (println 
      (map #(select-keys % [:text :user])
              (statuses-mentions-timeline :oauth-creds my-creds
                                           :params {:screen-name "abotcalledquest"}
                                           :callbacks (SyncSingleCallback. 
                                                               response-return-body
                                                               response-throw-error
                                                               exception-rethrow))))))

Turns out the user has a lot of info. We want to get the user’s id only. There’s a couple of ways of getting at nested values in maps in Clojure (don’t sleep on that arrow, we’ll come back to it!) Let’s use get-in to pull the user id out here.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
(defn -main
  []
  (dorun
      (println 
      (map #(get-in % [:user :id_str])
              (statuses-mentions-timeline :oauth-creds my-creds
                                           :params {:screen-name "abotcalledquest"}
                                           :callbacks (SyncSingleCallback. 
                                                               response-return-body
                                                               response-throw-error
                                                               exception-rethrow))))))

This works, but we need both the user id and the tweet’s text. Let’s go ahead and pull the response formatting step out into its own function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
(defn extractTweetInfo
  [tweetMap]
  {:tweet (:text tweetMap), :user (get-in tweetMap [:user :id_str])})

(defn -main
  []
  (dorun
      (println 
      (map extractTweetInfo
              (statuses-mentions-timeline :oauth-creds my-creds
                                           :params {:screen-name "abotcalledquest"}
                                           :callbacks (SyncSingleCallback. 
                                                               response-return-body
                                                               response-throw-error
                                                               exception-rethrow))))))

Ok, now we’ve got all the mentions, but we are building a bot of discerning taste here. We want to filter just the tweets that include a certain line of text. Java, which Clojure compiles to, has a fine method for detecting substrings already. Calling Java from Clojure is a simple matter of adding dots. We’ll bring back our anonymous function syntax to make this work:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
(defn -main
  []
  (dorun
      (println
      (filter #(.contains (:tweet %) "can i kick it")
              (map extractTweetInfo
              (statuses-mentions-timeline :oauth-creds my-creds
                                               :params {:screen-name "abotcalledquest"}
                                               :callbacks (SyncSingleCallback. 
                                                             response-return-body 
                                                             response-throw-error 
                                                             exception-rethrow))))

That anonymous function looks a bit backwards doesn’t it? It’s hard to read it from the inside out. Enter the arrow! The function being passed to filter can be rewritten as #(-> % :tweet (.contains "can i kick it")) using the threading macro. Like Jessica, I find this very exciting! Turns out we can do (almost) the same thing to the entire method & it reads a lot like how you’d describe what we doing in English:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
(defn -main
  []
  (dorun
    (->> (statuses-mentions-timeline :oauth-creds my-creds
                                   :params {:screen-name "abotcalledquest"}
                                   :callbacks (SyncSingleCallback. response-return-body  
                                                                   response-throw-error  
                                                                   exception-rethrow))
         (map extractTweetInfo)
         (filter #(-> % :tweet (.contains "can i kick it")))
         (println))))

Of course, not to respond would be très rude. We’ll need to change our app permission to allow writes, which may be more or less easy depending on your current level of Twitter phone number authentication. I had to set up a freakin Google Voice number, oy. Once that’s handled, add a reply method and map over the result set with it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
(defn extractTweetInfo
  [tweetMap]
  {:tweet (clojure.string/lower-case (:text tweetMap)), :tweet_id (:id_str tweetMap),
                                      :user (get-in tweetMap [:user :id_str]),
                                      :screen-name (get-in tweetMap [:user :screen_name])})

(defn replyToTweet
  [tweetMap]
  (statuses-update :oauth-creds my-creds
                   :params {:status (str "@" (:screen-name tweetMap) " yes you can"),
                            :in_reply_to_status_id (:tweet_id tweetMap)}
                            :callbacks (SyncSingleCallback. response-return-body
                                                            response-throw-error
                                                            exception-rethrow)))

(defn -main
  []
  (dorun
    (->> (statuses-mentions-timeline :oauth-creds my-creds
                                   :params {:screen-name "abotcalledquest"}
                                   :callbacks (SyncSingleCallback. response-return-body
                                                                   response-throw-error
                                                                   exception-rethrow))
         (map extractTweetInfo)
         (filter #(-> % :tweet (.contains "can i kick it")))
         (map replyToTweet)
         (println))))

We can now ask our bot the fateful question, and upon running our program …

successful tweet

Hooray!

…kind of. We can’t run it again because it pulls & responds to all mentions, whether already responded to or not, and Twitter rejects duplicate tweets. Even if we fix that, we have to constantly poll for new Tweets, and we’ll surely run afoul of Twitter’s rate limits very quickly. Next blog-time, we’ll make the big switch to use the streaming API. Spoiler alert: THAR BE DRAGONS. SCALY, MUTABLE STATE DRAGONS.