Facebook, Clojure, and REST requests

How long does a facebook pull take [on X network connection] ?


DISCLAIMER: This post is based on a previous one which utilizes the clojure json and clj-http client libraries.  If you don't understand that post and don't have the source working effectively, the getUrl method in this post won't have any meaning to you.  Nevertheless, you can learn how to use the "repeatedly" function to benchmark from this post.

1) First, lets define a function that TIMES a function :

;;this taken conveniently from the clojure list-serv.

(defn time1 [f] ""
    (let [starttime (System/nanoTime)]
          (f)
          (/ (- (System/nanoTime) starttime) 1e9))) 


2) Now that we can time a single function.... How do we benchmark this function by running it, say, 10 times ?

(repeatedly 10 #(time1 f1)) ;;where f1 is my function I want to benchmark.

3) Finally, we can define a function that runs off to facebook and pulls my data :


(defn f1 [] "" (getUrl "https://graph.facebook.com/me/home?access_token=AAAAAAITEghMBAHMwoRFdfiUF6UYeOEU1BeDeJuh7qMun0qWG83l814tfF31e3vZC2sdkB84wxZCdgZCuW4S0HBZCln0yivkATB2dgU1lflXf9Vuwv25n"))

TADAAAAAA

4) Now, tie it all together :


user=> (repeatedly 10 #(time1 f1))
(2.043743 2.464968 1.746619 2.5188 2.619898 1.871332 1.82722 1.768416 2.038544 1.775063)
user=> (reduce + (repeatedly 10 #(time1 f1))
18.713732999999998

So what have we learned ? 


That its really damn easy to wrap functions in other functions in Clojure :)

Oh, and also ----- that [at least in this case] it takes ~ 1.87 seconds to grab the text contents of a facebook wall page [assuming that clojure's http client isnt any kind of bottleneck].   Ive tried this for other full pages returned by facebook - and its more or less in the same range.  Of course, we could try the same with curl :

Users-MacBook-Pro-17:~ jayunit100$ time curl https://graph.facebook.com/me/home?access_token=AAAAAAITEghMBAHMwoRFdfiUF6UYeOEU1BeDeJuh7qMun0qWG83l814tfF31e3vZC2sdkB84wxZCdgZCuW4S0HBZCln0yivkATB2dgU1lflXf9Vuwv25n

real 0m1.768s

And get a similar result :) .... So ... that corroborates the idea that the crawling speed from clojure's http client is likely a pretty good reflection of the speed using another language/method for making web requests.

For an ad hoc test of a megabyte or larger download speeds, you can use the urls here : http://www.sportsci.com/topics2/AVI_files/ [or any other media site].