Skip to content

GitLab

Last edited by Dan McQuillan
Page history

lab 17

is52027c lab 17

apis

  • step 0 - consuming an api
  • step 1 - creating a simple json api
  • step 2 - using flask-restful to create an api

step 0: consuming apis

note that step-0 has been updated to reflect changes in urllib in python 3

using the OpenWeatherMap api

accessing with python

  • we will test the api in the python shell. type 'python' at the command line to enter the shell.

  • create your url as a string variable in your shell:

      url = "http://api.openweathermap.org/data/2.5/weather?q=London%2C%20UK&units=metric&appid=<YOUR API KEY HERE>"

  • note the location "London, UK" is url encoded in the url string

  • import urlopen from urllib.request so you can request the url (see https://docs.python.org/dev/tutorial/stdlib.html#internet-access)

      from urllib.request import urlopen

  • read the data returned by the url

      data = urlopen(url).read()

  • note that python is making an http request so the data is returned as binary data; to use it we need to decode it according to the charset returned by the server (which will usually be utf-8)

      data = data.decode(data.headers.get_content_charset())

parsing the data

  • if you print this out you will see it is a long string in what looks like a json format i.e. it is serialised json.

  • so we want to convert it back to json which we can do with the python function json.loads

  • import the json library

      import json

  • parse the data in to json

      weather = json.loads(data)

  • we want to check the structure of this data, so we can 'pretty print' the json data using json.dumps

      print(json.dumps(weather, indent=4, sort_keys=True))

accesssing the data

  • now you can access the json data structure

  • pay careful attention to which parts are a python dictionary (most of it) and which of those elements include lists

  • for example the current temperature in London is accessed as

      weather['main']['temp']

  • whereas the 'weather' element itself has a list structure, so the description is accessed as

      weather['weather'][0]['description']

  • you now have access to the current weather in a form you an use in your code

  • a call to the api will always return the latest up-to-date weather as made available by openweathermap

using apis

  • there are some apis which you can use without an api key but most will require you to register
  • the most common format for the data is json
  • you need to read the api develop docs to see the details of how to interrogate the api

step 1: simple api

using jsonify

  • remember from step 0 that the most common format for returning api data is json

  • we will create a simple api for our mytwits flask app using a flask function called jsonify http://flask.pocoo.org/docs/0.12/api/#flask.json.jsonify

  • this will directly return your json structure to the browser

  • to use jsonify you will need to import it i.e.

      from flask import jsonify

db -> api

  • a simple way to enable an api is to write a view which returns data directly from your db queries as json

  • we could use json.dumps, which serialises a python object as a json formatted stream https://docs.python.org/3.5/library/json.html

  • but jsonify adds the required http headers / mime types

  • add the following view to your app (assuming you have a db method like get_all_twits):

      @app.route('/api')
  def api():
  	twits = db.get_all_twits()
  	return jsonify({'twits':twits})

curl

  • we will be using curl to check returns from our apis

  • install curl on your vs

      sudo apt-get update
  sudo apt-get upgrade
  sudo apt-get install curl

  • while your app is running, connect to your virtual server via another terminal

  • test your api by connecting to it using curl

      curl -i localhost:8000/api

add api call to return data for a user

  • extend your api to return only the twits for a specific user

  • you will need to pass the username via the url

  • and call the appropriate db method

  • remember that you can have a route for a view that contains a variable e.g.

      	@app.route('/api/<username>')

step 2: using flask-restful

  • now we'll create a fulled-specced api using the flask-restful extension https://flask-restful.readthedocs.io/en/latest/
  • this requires a bit more set up but rewards us with more powerful functionality that handles the full range of RESTful HTTP methods
  • there is a template for this in the repo called 'mytwits_mysqli_template.py'
  • you should be able to see from the template where to add the code referred to below

flask-restful setup

basic steps

  • you will see from the flask-restful quickstart that we need 3 basic steps to implement a flask-restful api

  • initialise our api on the app

      api = Api(app)

  • creating our api resource class as something like this;

      class TwitsApi(Resource):
      def get(self):
  	return  db.get_all_twits()

  • add the resource to a route

      api.add_resource(TwitsApi,'/api')

no jsonify, just dictionaries

flask-restful will deal with converting the data to json and adding the correct headers, so we don't need to use jsonify.

however, to take advantage of flask-restful we need to pass it our data in the form of dictionaries. in our previous use of pymysql in mytwits_mysql.py we used the default cursor which returns the results as tuples. however, we can force pymysql to return the sql results as a python dictionary using pymysql.cursors.DictCursor (see the bottom of the pymysql api ref on cursors https://pymysql.readthedocs.io/en/latest/modules/cursors.html).

so for queries that we want to pass back to the api we replace

    with self.db.cursor() as cursor:

with

    with self.db.cursor(pymysql.cursors.DictCursor) as cursor:

( Note that this also means tweaking our jinja templates for the html output, where we will need to replace

  <div class="col-md-4">{{ twit[0] }}</div>

with

<div class="col-md-4">{{ twit['username'] }}</div>

)

data formatting and argument parsing

our api will go beyond the simplest flask-restful example in two ways

  1. using data formatting to deal with objects such as datetime objects
  2. using argument parsing to deal with request data validation

the validation is bsaically the same as we use for validating user form input, but here we are validating api requests.

data formatting

  • to format the data we will use the fields module and the @marshal_with() decorator

  • they are described in the flask-restful quickstart: https://flask-restful.readthedocs.io/en/latest/quickstart.html#data-formatting

  • first we need to import them

      from flask_restful import fields, marshal_with

  • we provide a set of resource fields

      #----resource fields to filter the output for flask-restful
  resource_fields = {
      'twit_id': fields.Integer,
      'twit': fields.String,
      'user_id': fields.Integer,
      'created_at': fields.DateTime(dt_format='rfc822')
  }

  • we use these to format the output using the decorator

      class TwitsApi(Resource):
      @marshal_with(resource_fields)
      def get(self):
  	return  db.get_all_twits()

  • in our case this means we can handle the datetime objects; otherwise we would get a message saying "datetime.datetime is not JSON serializable"

argument parsing

  • we need to import the reqparse module

      from flask_restful import reqparse

  • this module is described in the flask-restful quickstart: https://flask-restful.readthedocs.io/en/latest/quickstart.html#argument-parsing

  • as well as checking the input it can be used to provide helpful error messages to the user

      #----parse and verify the api inputs
  parser = reqparse.RequestParser()
  parser.add_argument('twit', type=str, help='the text of the twit; must be a string')
  parser.add_argument('twit_id', type=int, help='the id of the twit; must be an integer')
  parser.add_argument('user_id', type=int, help='the id of the twit; must be an integer')

  • this is used below when we are adding a twit via the api

adding twits via the api

  • flask-restful gives us easy access to the 4 HTTP actions; GET, POST, PUT, DELETE

  • for example, to add a twit we simply define a post method on our TwitsApi resource

  • we also use reqparse to vaildate the input

      class TwitsApi(Resource):
      @marshal_with(resource_fields)
      def get(self):
  	return  db.get_all_twits()

  • add a post method to the same class:

          @marshal_with(resource_fields)
      def post(self):
  	args = parser.parse_args()
  	twit = args['twit']
  	user_id = args['user_id']
  	db.add_twit(twit,user_id)
  	return  db.get_all_twits()

  • now a correctly formatted POST request to the api will add a new twit

testing

  • if you add the above code snippets in the correct parts of the template you should be able to get twits using curl

      curl -i localhost:8000/api

  • flask-restful recognise a twit posted as form data

  • see the curl manual for more details on how to emulate a web form: https://curl.haxx.se/docs/httpscripting.html#POST

      curl -d "twit='posted using a form'&user_id=1&submit=SUBMIT" localhost:8000/api

  • flask-restful will also recognise a twit posted as json:

      curl -H "Content-Type: application/json" -X POST -d '{"twit":"posted using json","user_id":1}' http://localhost:8000/api

extend to PUT and DELETE

  • we can extend the api to include PUT and DELETE as well

  • PUT will be used to modify (edit) twits and DELETE to delete them(!)

  • these methods will require the id of the twit we want to act on

  • so we can create another flask-restful resourcei...

      class TwitsIdApi(Resource):

  • and connect that to a path that includes a twit id

     api.add_resource(TwitsIdApi,'/api/<int:twit_id>')

  • use the db methods you previously implemented (or the ones from my repo code) to implement PUT and DELETE as part of the api

  • you can also test these HTTP methods using curl; see the quickstart full example for examples of the correct curl commands: https://flask-restful.readthedocs.io/en/latest/quickstart.html#full-example

  • e.g.

      curl -d "twit=changed this" localhost:8000/api/7 -X PUT -v
  
  curl localhost:8000/api/7 -X DELETE -v

extension