is52027c lab 17
apis
- step 0 - consuming an api
- step 1 - creating a simple json api
- step 2 - using flask-restful to create an api
step 0: consuming apis
note that step-0 has been updated to reflect changes in urllib in python 3
using the OpenWeatherMap api
- to use the open weather map, sign up at http://openweathermap.org/appid
- then you can get an api key from https://home.openweathermap.org/api_keys
- the data returned by the current weather api is described at https://openweathermap.org/current
- note that according to the docs "Activation of an API key for Free and Startup accounts takes 10 minutes"
accessing with python
-
we will test the api in the python shell. type 'python' at the command line to enter the shell.
-
create your url as a string variable in your shell:
url = "http://api.openweathermap.org/data/2.5/weather?q=London%2C%20UK&units=metric&appid=<YOUR API KEY HERE>"
-
note the location "London, UK" is url encoded in the url string
-
import urlopen from urllib.request so you can request the url (see https://docs.python.org/dev/tutorial/stdlib.html#internet-access)
from urllib.request import urlopen
-
read the data returned by the url
data = urlopen(url).read()
-
note that python is making an http request so the data is returned as binary data; to use it we need to decode it according to the charset returned by the server (which will usually be utf-8)
data = data.decode(data.headers.get_content_charset())
parsing the data
-
if you print this out you will see it is a long string in what looks like a json format i.e. it is serialised json.
-
so we want to convert it back to json which we can do with the python function json.loads
-
import the json library
import json
-
parse the data in to json
weather = json.loads(data)
-
we want to check the structure of this data, so we can 'pretty print' the json data using json.dumps
print(json.dumps(weather, indent=4, sort_keys=True))
accesssing the data
-
now you can access the json data structure
-
pay careful attention to which parts are a python dictionary (most of it) and which of those elements include lists
-
for example the current temperature in London is accessed as
weather['main']['temp']
-
whereas the 'weather' element itself has a list structure, so the description is accessed as
weather['weather'][0]['description']
-
you now have access to the current weather in a form you an use in your code
-
a call to the api will always return the latest up-to-date weather as made available by openweathermap
using apis
- there are some apis which you can use without an api key but most will require you to register
- the most common format for the data is json
- you need to read the api develop docs to see the details of how to interrogate the api
step 1: simple api
using jsonify
-
remember from step 0 that the most common format for returning api data is json
-
we will create a simple api for our mytwits flask app using a flask function called jsonify http://flask.pocoo.org/docs/0.12/api/#flask.json.jsonify
-
this will directly return your json structure to the browser
-
to use jsonify you will need to import it i.e.
from flask import jsonify
db -> api
-
a simple way to enable an api is to write a view which returns data directly from your db queries as json
-
we could use json.dumps, which serialises a python object as a json formatted stream https://docs.python.org/3.5/library/json.html
-
but jsonify adds the required http headers / mime types
-
add the following view to your app (assuming you have a db method like get_all_twits):
@app.route('/api') def api(): twits = db.get_all_twits() return jsonify({'twits':twits})
curl
-
we will be using curl to check returns from our apis
-
install curl on your vs
sudo apt-get update sudo apt-get upgrade sudo apt-get install curl
-
while your app is running, connect to your virtual server via another terminal
-
test your api by connecting to it using curl
curl -i localhost:8000/api
add api call to return data for a user
-
extend your api to return only the twits for a specific user
-
you will need to pass the username via the url
-
and call the appropriate db method
-
remember that you can have a route for a view that contains a variable e.g.
@app.route('/api/<username>')
step 2: using flask-restful
- now we'll create a fulled-specced api using the flask-restful extension https://flask-restful.readthedocs.io/en/latest/
- this requires a bit more set up but rewards us with more powerful functionality that handles the full range of RESTful HTTP methods
- there is a template for this in the repo called 'mytwits_mysqli_template.py'
- you should be able to see from the template where to add the code referred to below
flask-restful setup
-
install flask-restful on your vs
pip install --user flask-restful
-
the main building blocks of a flask-restful api are resources
-
creating an api class based on flask-restful's Resource class means we can get easy access to all the HTTP methods
-
you can see how this works at https://flask-restful.readthedocs.io/en/latest/quickstart.html#resourceful-routing
basic steps
-
you will see from the flask-restful quickstart that we need 3 basic steps to implement a flask-restful api
-
initialise our api on the app
api = Api(app)
-
creating our api resource class as something like this;
class TwitsApi(Resource): def get(self): return db.get_all_twits()
-
add the resource to a route
api.add_resource(TwitsApi,'/api')
no jsonify, just dictionaries
flask-restful will deal with converting the data to json and adding the correct headers, so we don't need to use jsonify.
however, to take advantage of flask-restful we need to pass it our data in the form of dictionaries. in our previous use of pymysql in mytwits_mysql.py we used the default cursor which returns the results as tuples. however, we can force pymysql to return the sql results as a python dictionary using pymysql.cursors.DictCursor (see the bottom of the pymysql api ref on cursors https://pymysql.readthedocs.io/en/latest/modules/cursors.html).
so for queries that we want to pass back to the api we replace
with self.db.cursor() as cursor:
with
with self.db.cursor(pymysql.cursors.DictCursor) as cursor:
( Note that this also means tweaking our jinja templates for the html output, where we will need to replace
<div class="col-md-4">{{ twit[0] }}</div>
with
<div class="col-md-4">{{ twit['username'] }}</div>
)
data formatting and argument parsing
our api will go beyond the simplest flask-restful example in two ways
- using data formatting to deal with objects such as datetime objects
- using argument parsing to deal with request data validation
the validation is bsaically the same as we use for validating user form input, but here we are validating api requests.
data formatting
-
to format the data we will use the fields module and the @marshal_with() decorator
-
they are described in the flask-restful quickstart: https://flask-restful.readthedocs.io/en/latest/quickstart.html#data-formatting
-
first we need to import them
from flask_restful import fields, marshal_with
-
we provide a set of resource fields
#----resource fields to filter the output for flask-restful resource_fields = { 'twit_id': fields.Integer, 'twit': fields.String, 'user_id': fields.Integer, 'created_at': fields.DateTime(dt_format='rfc822') }
-
we use these to format the output using the decorator
class TwitsApi(Resource): @marshal_with(resource_fields) def get(self): return db.get_all_twits()
-
in our case this means we can handle the datetime objects; otherwise we would get a message saying "datetime.datetime is not JSON serializable"
argument parsing
-
we need to import the reqparse module
from flask_restful import reqparse
-
this module is described in the flask-restful quickstart: https://flask-restful.readthedocs.io/en/latest/quickstart.html#argument-parsing
-
as well as checking the input it can be used to provide helpful error messages to the user
#----parse and verify the api inputs parser = reqparse.RequestParser() parser.add_argument('twit', type=str, help='the text of the twit; must be a string') parser.add_argument('twit_id', type=int, help='the id of the twit; must be an integer') parser.add_argument('user_id', type=int, help='the id of the twit; must be an integer')
-
this is used below when we are adding a twit via the api
adding twits via the api
-
flask-restful gives us easy access to the 4 HTTP actions; GET, POST, PUT, DELETE
-
for example, to add a twit we simply define a post method on our TwitsApi resource
-
we also use reqparse to vaildate the input
class TwitsApi(Resource): @marshal_with(resource_fields) def get(self): return db.get_all_twits()
-
add a post method to the same class:
@marshal_with(resource_fields) def post(self): args = parser.parse_args() twit = args['twit'] user_id = args['user_id'] db.add_twit(twit,user_id) return db.get_all_twits()
-
now a correctly formatted POST request to the api will add a new twit
testing
-
if you add the above code snippets in the correct parts of the template you should be able to get twits using curl
curl -i localhost:8000/api
-
flask-restful recognise a twit posted as form data
-
see the curl manual for more details on how to emulate a web form: https://curl.haxx.se/docs/httpscripting.html#POST
curl -d "twit='posted using a form'&user_id=1&submit=SUBMIT" localhost:8000/api
-
flask-restful will also recognise a twit posted as json:
curl -H "Content-Type: application/json" -X POST -d '{"twit":"posted using json","user_id":1}' http://localhost:8000/api
extend to PUT and DELETE
-
we can extend the api to include PUT and DELETE as well
-
PUT will be used to modify (edit) twits and DELETE to delete them(!)
-
these methods will require the id of the twit we want to act on
-
so we can create another flask-restful resourcei...
class TwitsIdApi(Resource):
-
and connect that to a path that includes a twit id
api.add_resource(TwitsIdApi,'/api/<int:twit_id>')
-
use the db methods you previously implemented (or the ones from my repo code) to implement PUT and DELETE as part of the api
-
you can also test these HTTP methods using curl; see the quickstart full example for examples of the correct curl commands: https://flask-restful.readthedocs.io/en/latest/quickstart.html#full-example
-
e.g.
curl -d "twit=changed this" localhost:8000/api/7 -X PUT -v curl localhost:8000/api/7 -X DELETE -v
extension
- study the flask-restful docs https://flask-restful.readthedocs.io/en/latest/
- consider how to extend your api
- implement your changes