{"meta":{"title":"Rendering data as graphs","intro":"Learn how to visualize the programming languages from your repository using the D3.js library and Ruby Octokit.","product":"REST API","breadcrumbs":[{"href":"/en/rest","title":"REST API"},{"href":"/en/rest/guides","title":"Guides"},{"href":"/en/rest/guides/rendering-data-as-graphs","title":"Rendering data as graphs"}],"documentType":"article"},"body":"# Rendering data as graphs\n\nLearn how to visualize the programming languages from your repository using the D3.js library and Ruby Octokit.\n\nIn this guide, we're going to use the API to fetch information about repositories\nthat we own, and the programming languages that make them up. Then, we'll\nvisualize that information in a couple of different ways using the [D3.js](https://d3js.org/) library. To\ninteract with the GitHub API, we'll be using the excellent Ruby library, [Octokit](https://github.com/octokit/octokit.rb).\n\nIf you haven't already, you should read the [Basics of Authentication](/en/apps/oauth-apps/building-oauth-apps/authenticating-to-the-rest-api-with-an-oauth-app)\nguide before starting this example. You can find the complete source code for this project in the [platform-samples](https://github.com/github/platform-samples/tree/master/api/ruby/rendering-data-as-graphs) repository.\n\nLet's jump right in!\n\n## Setting up an OAuth app\n\nFirst, [register a new application](https://github.com/settings/applications/new) on GitHub. Set the main and callback\nURLs to `http://localhost:4567/`. As [before](/en/apps/oauth-apps/building-oauth-apps/authenticating-to-the-rest-api-with-an-oauth-app), we're going to handle authentication for the API by\nimplementing a Rack middleware using [sinatra-auth-github](https://rubygems.org/gems/sinatra_auth_github):\n\n```ruby\nrequire 'sinatra/auth/github'\n\nmodule Example\n  class MyGraphApp < Sinatra::Base\n    # !!! DO NOT EVER USE HARD-CODED VALUES IN A REAL APP !!!\n    # Instead, set and test environment variables, like below\n    # if ENV['GITHUB_CLIENT_ID'] && ENV['GITHUB_CLIENT_SECRET']\n    #  CLIENT_ID        = ENV['GITHUB_CLIENT_ID']\n    #  CLIENT_SECRET    = ENV['GITHUB_CLIENT_SECRET']\n    # end\n\n    CLIENT_ID = ENV['GH_GRAPH_CLIENT_ID']\n    CLIENT_SECRET = ENV['GH_GRAPH_SECRET_ID']\n\n    enable :sessions\n\n    set :github_options, {\n      :scopes    => \"repo\",\n      :secret    => CLIENT_SECRET,\n      :client_id => CLIENT_ID,\n      :callback_url => \"/\"\n    }\n\n    register Sinatra::Auth::Github\n\n    get '/' do\n      if !authenticated?\n        authenticate!\n      else\n        access_token = github_user[\"token\"]\n      end\n    end\n  end\nend\n```\n\nSet up a similar *config.ru* file as in the previous example:\n\n```ruby\nENV['RACK_ENV'] ||= 'development'\nrequire \"rubygems\"\nrequire \"bundler/setup\"\n\nrequire File.expand_path(File.join(File.dirname(__FILE__), 'server'))\n\nrun Example::MyGraphApp\n```\n\n## Fetching repository information\n\nThis time, in order to talk to the GitHub API, we're going to use the [Octokit\nRuby library](https://github.com/octokit/octokit.rb). This is much easier than directly making a bunch of\nREST calls. Plus, Octokit was developed by a GitHubber, and is actively maintained,\nso you know it'll work.\n\nAuthentication with the API via Octokit is easy. Just pass your login\nand token to the `Octokit::Client` constructor:\n\n```ruby\nif !authenticated?\n  authenticate!\nelse\n  octokit_client = Octokit::Client.new(:login => github_user.login, :oauth_token => github_user.token)\nend\n```\n\nLet's do something interesting with the data about our repositories. We're going\nto see the different programming languages they use, and count which ones are used\nmost often. To do that, we'll first need a list of our repositories from the API.\nWith Octokit, that looks like this:\n\n```ruby\nrepos = client.repositories\n```\n\nNext, we'll iterate over each repository, and count the language that GitHub\nassociates with it:\n\n```ruby\nlanguage_obj = {}\nrepos.each do |repo|\n  # sometimes language can be nil\n  if repo.language\n    if !language_obj[repo.language]\n      language_obj[repo.language] = 1\n    else\n      language_obj[repo.language] += 1\n    end\n  end\nend\n\nlanguages.to_s\n```\n\nWhen you restart your server, your web page should display something\nthat looks like this:\n\n```ruby\n{\"JavaScript\"=>13, \"PHP\"=>1, \"Perl\"=>1, \"CoffeeScript\"=>2, \"Python\"=>1, \"Java\"=>3, \"Ruby\"=>3, \"Go\"=>1, \"C++\"=>1}\n```\n\nSo far, so good, but not very human-friendly. A visualization\nwould be great in helping us understand how these language counts are distributed. Let's feed\nour counts into D3 to get a neat bar graph representing the popularity of the languages we use.\n\n## Visualizing language counts\n\nD3.js, or just D3, is a comprehensive library for creating many kinds of charts, graphs, and interactive visualizations.\nUsing D3 in detail is beyond the scope of this guide, but for a good introductory article,\ncheck out [D3 for Mortals](http://recursion.org/d3-for-mere-mortals/).\n\nD3 is a JavaScript library, and likes working with data as arrays. So, let's convert our Ruby hash into\na JSON array for use by JavaScript in the browser.\n\n```ruby\nlanguages = []\nlanguage_obj.each do |lang, count|\n  languages.push :language => lang, :count => count\nend\n\nerb :lang_freq, :locals => { :languages => languages.to_json}\n```\n\nWe're simply iterating over each key-value pair in our object and pushing them into\na new array. The reason we didn't do this earlier is because we didn't want to iterate\nover our `language_obj` object while we were creating it.\n\nNow, *lang\\_freq.erb* is going to need some JavaScript to support rendering a bar graph.\nFor now, you can just use the code provided here, and refer to the resources linked above\nif you want to learn more about how D3 works:\n\n```html\n<!DOCTYPE html>\n<meta charset=\"utf-8\">\n<html>\n  <head>\n    <script src=\"//cdnjs.cloudflare.com/ajax/libs/d3/3.0.1/d3.v3.min.js\"></script>\n    <style>\n    svg {\n      padding: 20px;\n    }\n    rect {\n      fill: #2d578b\n    }\n    text {\n      fill: white;\n    }\n    text.yAxis {\n      font-size: 12px;\n      font-family: Helvetica, sans-serif;\n      fill: black;\n    }\n    </style>\n  </head>\n  <body>\n    <p>Check this sweet data out:</p>\n    <div id=\"lang_freq\"></div>\n\n  </body>\n  <script>\n    var data = <%= languages %>;\n\n    var barWidth = 40;\n    var width = (barWidth + 10) * data.length;\n    var height = 300;\n\n    var x = d3.scale.linear().domain([0, data.length]).range([0, width]);\n    var y = d3.scale.linear().domain([0, d3.max(data, function(datum) { return datum.count; })]).\n      rangeRound([0, height]);\n\n    // add the canvas to the DOM\n    var languageBars = d3.select(\"#lang_freq\").\n      append(\"svg:svg\").\n      attr(\"width\", width).\n      attr(\"height\", height);\n\n    languageBars.selectAll(\"rect\").\n      data(data).\n      enter().\n      append(\"svg:rect\").\n      attr(\"x\", function(datum, index) { return x(index); }).\n      attr(\"y\", function(datum) { return height - y(datum.count); }).\n      attr(\"height\", function(datum) { return y(datum.count); }).\n      attr(\"width\", barWidth);\n\n    languageBars.selectAll(\"text\").\n      data(data).\n      enter().\n      append(\"svg:text\").\n      attr(\"x\", function(datum, index) { return x(index) + barWidth; }).\n      attr(\"y\", function(datum) { return height - y(datum.count); }).\n      attr(\"dx\", -barWidth/2).\n      attr(\"dy\", \"1.2em\").\n      attr(\"text-anchor\", \"middle\").\n      text(function(datum) { return datum.count;});\n\n    languageBars.selectAll(\"text.yAxis\").\n      data(data).\n      enter().append(\"svg:text\").\n      attr(\"x\", function(datum, index) { return x(index) + barWidth; }).\n      attr(\"y\", height).\n      attr(\"dx\", -barWidth/2).\n      attr(\"text-anchor\", \"middle\").\n      text(function(datum) { return datum.language;}).\n      attr(\"transform\", \"translate(0, 18)\").\n      attr(\"class\", \"yAxis\");\n  </script>\n</html>\n```\n\nPhew! Again, don't worry about what most of this code is doing. The relevant part\nhere is a line way at the top--`var data = <%= languages %>;`--which indicates\nthat we're passing our previously created `languages` array into ERB for manipulation.\n\nAs the \"D3 for Mortals\" guide suggests, this isn't necessarily the best use of\nD3. But it does serve to illustrate how you can use the library, along with Octokit,\nto make some really amazing things.\n\n## Combining different API calls\n\nNow it's time for a confession: the `language` attribute within repositories\nonly identifies the \"primary\" language defined. That means that if you have\na repository that combines several languages, the one with the most bytes of code\nis considered to be the primary language.\n\nLet's combine a few API calls to get a *true* representation of which language\nhas the greatest number of bytes written across all our code. A [treemap](https://www.d3-graph-gallery.com/treemap.html)\nshould be a great way to visualize the sizes of our coding languages used, rather\nthan simply the count. We'll need to construct an array of objects that looks\nsomething like this:\n\n```json\n[ { \"name\": \"language1\", \"size\": 100},\n  { \"name\": \"language2\", \"size\": 23}\n  ...\n]\n```\n\nSince we already have a list of repositories above, let's inspect each one, and\ncall the [GET /repos/{owner}/{repo}/languages endpoint](/en/rest/repos/repos#list-repository-languages):\n\n```ruby\nrepos.each do |repo|\n  repo_name = repo.name\n  repo_langs = octokit_client.languages(\"#{github_user.login}/#{repo_name}\")\nend\n```\n\nFrom there, we'll cumulatively add each language found to a list of languages:\n\n```ruby\nrepo_langs.each do |lang, count|\n  if !language_obj[lang]\n    language_obj[lang] = count\n  else\n    language_obj[lang] += count\n  end\nend\n```\n\nAfter that, we'll format the contents into a structure that D3 understands:\n\n```ruby\nlanguage_obj.each do |lang, count|\n  language_byte_count.push :name => \"#{lang} (#{count})\", :count => count\nend\n\n# some mandatory formatting for D3\nlanguage_bytes = [ :name => \"language_bytes\", :elements => language_byte_count]\n```\n\n(For more information on D3 tree map magic, check out [this simple tutorial](/en/rest/repos/repos#list-repository-languages).)\n\nTo wrap up, we pass this JSON information over to the same ERB template:\n\n```ruby\nerb :lang_freq, :locals => { :languages => languages.to_json, :language_byte_count => language_bytes.to_json}\n```\n\nLike before, here's a bunch of JavaScript that you can drop\ndirectly into your template:\n\n```html\n<div id=\"byte_freq\"></div>\n<script>\n  var language_bytes = <%= language_byte_count %>\n  var childrenFunction = function(d){return d.elements};\n  var sizeFunction = function(d){return d.count;};\n  var colorFunction = function(d){return Math.floor(Math.random()*20)};\n  var nameFunction = function(d){return d.name;};\n\n  var color = d3.scale.linear()\n              .domain([0,10,15,20])\n              .range([\"grey\",\"green\",\"yellow\",\"red\"]);\n\n  drawTreemap(5000, 2000, '#byte_freq', language_bytes, childrenFunction, nameFunction, sizeFunction, colorFunction, color);\n\n  function drawTreemap(height,width,elementSelector,language_bytes,childrenFunction,nameFunction,sizeFunction,colorFunction,colorScale){\n\n      var treemap = d3.layout.treemap()\n          .children(childrenFunction)\n          .size([width,height])\n          .value(sizeFunction);\n\n      var div = d3.select(elementSelector)\n          .append(\"div\")\n          .style(\"position\",\"relative\")\n          .style(\"width\",width + \"px\")\n          .style(\"height\",height + \"px\");\n\n      div.data(language_bytes).selectAll(\"div\")\n          .data(function(d){return treemap.nodes(d);})\n          .enter()\n          .append(\"div\")\n          .attr(\"class\",\"cell\")\n          .style(\"background\",function(d){ return colorScale(colorFunction(d));})\n          .call(cell)\n          .text(nameFunction);\n  }\n\n  function cell(){\n      this\n          .style(\"left\",function(d){return d.x + \"px\";})\n          .style(\"top\",function(d){return d.y + \"px\";})\n          .style(\"width\",function(d){return d.dx - 1 + \"px\";})\n          .style(\"height\",function(d){return d.dy - 1 + \"px\";});\n  }\n</script>\n```\n\nEt voila! Beautiful rectangles containing your repo languages, with relative\nproportions that are easy to see at a glance. You might need to\ntweak the height and width of your treemap, passed as the first two\narguments to `drawTreemap` above, to get all the information to show up properly."}