Debugging - Build APIs You Won't Hate: Everyone and their dog wants an API, so you should probably learn how to build them (2014)

Build APIs You Won't Hate: Everyone and their dog wants an API, so you should probably learn how to build them (2014)

8. Debugging

8.1 Introduction

Debugging is the art of working out why something is broken, which can be pretty difficult in an API. In much of web development you are simply looking at what is output to the page, overusing var_dump() or checking the browsers console for JavaScript errors. Working with an API you are mostly just working with Requests and Responses, but you need to initiate these requests in a repeatable way, often with full control over all of the HTTP Headers, Body content, etc.

There are a few methods you can utilize for debugging:

· Command-line Debugging

· Browser Debugging

· Network Debugging

8.2 Command-line Debugging

Debugging via the command-line using tools like curl are a great option for some. They tout the benefits of being able to do it from inside a network firewall. Certainly this can be an option for debugging live servers, but for development purposes (which is what we are doing here) then using curl is just a lot of commands to remember for no reason.

1 $ curl -X POST http://localhost/places/fg345d/checkins --data @payload.json

It is not the most complicated way to initiate a request, but it is not the easiest. You will need to update that payload.json each time, or have a bunch of JSON in the CLI, and either way that is a pain in the backside when you have a lot of endpoints with lots of potential values.

8.3 Browser Debugging

Working in the browser is a great way to do things and developers are fairly used to it. Sadly most browsers can only really handle GET and POST requests by default, and a RESTful API requires PUT, DELETE, PATCH, etc too. A well built RESTful API will also require the use of HTTP Headers, which can be difficult to manipulate in a browser, as they are built to handle all of that for you.

HTTP Clients

Called “HTTP Clients” or “REST Client” interchangeably, these bits of software help perfectly with the job this book sets out to achieve: building non-trivial APIs. They allow you to format your HTTP Request through a convenient GUI, choosing the HTTP verb, adding headers, entering a body, etc then present the HTTP Response to you with formatting or in source view if you prefer. Many of these GUIs will let you save common requests or build “collections” much like a set of bookmarks, but for your endpoints and with all the correct headers and values.

These clients exist for Windows, OSX and Linux but one that has really stood out to me is a Chrome extension called Postman.

Postman HTTP Client, showing a collection and a sucessful JSON response

Postman HTTP Client, showing a collection and a sucessful JSON response

I have a collection which almost mirrors my Behat tests, and have at least one for each endpoint - some with more.

Using Postman I can develop “in the browser” and see errors easily, and keep changing things and clicking “Send” for as long as I have to to make it work. When I expect it to work I run the Behat scenario that covers the endpoint, and see if the tests are green. If Behat fails and the errors are not enough to resolve the problem then simply go back to Postman and try again.

Repeat until the endpoint “works”, and passes the test.

Debug Panel

The approach above works fine if the problem is one that you can see. Anything to do with a slow page return, silent fails, unexpected results, etc. all need more information, and to do that you probably need another extension.

RailsPanel - Chrome-only DevTool panel with logging and profiling for Ruby on Rails. (RailsCasts Video).

Clockwork - Chrome DevTool panel and standalone web app with logging and profiling for PHP.

Chrome Logger - Chrome Logger only for Python, PHP, Ruby, Node, .NET, CF and Go.

The first two are very similar and are the most feature filled, but the latter covers basic logging for the wider selection of languages.

Sure these examples are mostly Chrome. There are probably alternatives, but either way there is no harm in having Chrome as your “Development Browser” and keep on using your favorite for general browsing.

Clockwork showing the Laravel timeline in Chromium Browser

Clockwork showing the Laravel timeline in Chromium Browser

This timeline can be useful for working out where things are slowing down. Define your own events to see where the time is going.

Seeing logs in this panel is another benefit, and it helps keep you from switching back to the console all the time to catch the output of your logs via tail -f. Certainly you should be in the command line anyway, but constantly hitting Alt+Tab can cause distractions which slow you down.

For those of you who normally debug with var_dump() or breakpoints, you could simply use Clockwork/RailsPanel/Chrome Logger to do it, and see it in the panel - leaving your output untouched and avoiding tricky setup with IDE or other GUI programs.

CheckinTransformer using Fractal, with added Logging


1 <?php namespace App\Transformer;

2

3 use Checkin;

4 use Log;

5

6 use League\Fractal\TransformerAbstract;

7

8 classCheckinTransformerextends TransformerAbstract

9 {

10 /**

11 * List of resources possible to embed via this processor

12 *

13 * @var array

14 */

15 protected $availableEmbeds = [

16 'place',

17 'user',

18 ];

19

20 /**

21 * Turn this item object into a generic array

22 *

23 * @return array

24 */

25 publicfunction transform(Checkin $checkin)

26 {

27 return [

28 'id' => (int) $checkin->id,

29 'created_at' => (string) $checkin->created_at,

30 ];

31 }

32

33 /**

34 * Embed Place

35 *

36 * @return League\Fractal\Resource\Item

37 */

38 publicfunction embedPlace(Checkin $checkin)

39 {

40 $place = $checkin->place;

41

42 Log::info("Embedding place-{$place->id} into checkin-{$checkin->id}");

43

44 return $this->item($place, new PlaceTransformer);

45 }

46

47 /**

48 * Embed User

49 *

50 * @return League\Fractal\Resource\Item

51 */

52 publicfunction embedUser(Checkin $checkin)

53 {

54 $user = $checkin->user;

55

56 Log::info("Embedding user-{$user->id} into checkin-{$checkin->id}");

57

58 return $this->item($user, new UserTransformer);

59 }

60 }


That will look a little something like this:

Clockwork showing the Log in Chromium Browser

Clockwork showing the Log in Chromium Browser

You can log arrays and objects too:

Clockwork showing the Log in Chromium Browser

Clockwork showing the Log in Chromium Browser

If logging something cannot help you with a problem, you need to log more things. Eventually you will work it out.

8.4 Network Debugging

The previously mentioned approaches to debugging are very much about being in control: create a Request, see what happens with the Response. Sometimes you need to debug what is happening to your API when the requests are not completely in control. If your iPhone developer comes over and says “the API is broken” it can be hard to work out why.

If you know exactly what endpoint is being hit and what the error is (because the iPhone dev is pointing to some debug data on his XCode screen) then maybe you can fix it, but often you will need more insight before you can recreate it. Maybe it is not even a request that you can recreate easily (or at all) like anything to do with upload images as a PUT after getting them from the camera, or there are multiple requests that the iPhone app is executing in order using data from the previous requests.

Whatever the reason, sometimes you need to debug network activity to find out what is actually happening, by spying on the Request and getting the Response.

Charles

If these are non-production errors that you want to debug against your local API and development iOS devices (a.k.a the iPhone 4S you have not sold on eBay yet) then a great application is Charles.

Charles essentially acts as a HTTP proxy, which means stuff comes in, stuff goes out, and Charles can show you what that was. Beyond that it can rewrite headers and even let you modify the content of the request or response if you want to.

To set the basics of this up, you first need to know the internal network of your machine.

Network Settings on Mac OSX, showing local IP

Network Settings on Mac OSX, showing local IP

On your mobile device you will need to enable a HTTP Proxy. Enter your computers “local IP” in the “Proxy Server Address” field, and select port 8888 - the default Charles port.

Sample Charles HTTP Proxy settings on iOS7

Sample Charles HTTP Proxy settings on iOS7

This will forward all web traffic on to Charles, which (if it is running) will forward it on to its location.

As pointless as that might sound, the power comes in the options Charles has to offer. If we are intending to allow web traffic from our mobile device to the API on our development environment then at this point we are half way.

warning

Local v “Remote”

To allow Laravel (PHP’s) built in server to access this connection on OSX, you must start the server using the network address shown in the sharing section of system preferences. Choose Apple menu > System Preferences, and then click Sharing Below “Computer Name” you will see an address followed by “.local” To start the server simply use: $ php artisan serve –host=”Phils-MacBook-Air.local” I personally have Charles pointing to a Vagrant box, running on its own IP address with its own virtual host enabled. This is not something that the book will cover, but is certainly something you should look into doing.

To make dev-api.example.com mean something on your mobile device, enter a “Map Remote” rule in Charles.

Screenshot of Charles on OSX mapping dev-api.example.com

Screenshot of Charles on OSX mapping dev-api.example.com

As explained above Charles acts as a “man-in-the-middle”, re-routing traffic based on your rules. By saying dev-api.example.com should be routed to dev-api.example.com on your machine, you have given that hostname meaning on your mobile devices (or anything else talking to Charles on that port).

Now - so long as you are able to get a build of your mobile application pointing to dev-api.example.com - you will be able to click around the application, seeing requests and responses, with all of the according headers and values as you go.

You might not find yourself using Charles every day, or for a long time at the start as your HTTP Clients may be enough to debug problems, but having it available is certainly going to help you out at some point. Keep it in mind.

Wireshark is also handy for Linux/OSX users, and Fiddler is fun for Windows users.