Face Recognition using Javascript and Mashape

This is a tutorial on how you can use the Face Recognition API in a web application using Javascript.  The Face Recognition API makes it ridiculously easy to add face recognition capabilities to your app, whether it’s web, mobile, etc.  We will use photobooth.js to take pictures from our webcam. (Make sure you have one in your machine or plug one.)

There are 2 main things to remember when using the Face Recognition API.  You need to:

A.  “Train” the Face Recognition API, which means uploading a bunch of pictures that will constitute the “database” from which pictures will be recognized.

B.  “Recognize” a picture by uploading it to the Face Recognition API.

These will be the two main parts of this tutorial.  Let’s get started!

 

Train the Face Recognition API

1.  Get a Mashape account and key.  If you don’t have an account yet, you sign up here.  You need a Mashape account because we will use Mashape’s Test Console to upload pictures to Face Recognition to build our database.  Once you have signed up, you will be taken to your Mashape dashboard where you will have access to your Mashape keys,  as in the picture below:

image

 

2.  Go to the Face Recognition API page.  This page shows the API endpoints on the left, and their corresponding documentation and test console to the right.

What we’re interested right now is in creating an album where we can upload pictures to “train” the service.  You can create an album through the endpoint called “Create Album”.  You can click here to directly navigate to it in the page.

Think of a unique album name and hit “Test Endpoint”.  This will call the endpoint and return a response similar to below:

image

 

The response above indicates that we have successfully created an album in Face Recognition, and that we should remember/keep the values album and albumkey because we will be using it throughout the tutorial to refer to this album.  (Note: It’s probably obvious by now, but everything is hosted on the cloud – no need to maintain an on-premise setup).

3. Let’s now start “training” this album using the Train Album endpoint.  As you can see, one of the parameters in the Train Album endpoint requires us to upload pictures:

image

 

To take pictures in our webcam, we will use photobooth.js.  Click here to access the demo page where we will take our pictures.  We will also save the pictures in our drive so we can use them to upload/train the album later.

Note that photobooth.js will ask for our permission to use the webcam.  Just click “Allow”.

image

 

To take pictures, just click the camera icon to the right of the photo canvas.

image

It will take a picture every time you click. Take one picture and save it to your drive by right-clicking “Save Image As”.  Take three pictures of yourself and save it to your drive.  Then take one other picture of your buddy, as we will need this to differentiate the “entryid” in our album as required by the Train Album endpoint.

Once you’re done taking pictures, upload them to Train Album endpoint by filling out the required parameters, changing the “entryid” field for each person we want to upload (e.g. Entryid “Chris1” for Chris’ pictures and another entryid “Hazel1” for Hazel’s pictures.

image

 

 

 

You can upload one picture for each call so you have to do this for each upload.

(Note: You can also provide URLs pointing to your images.  For this tutorial we’ll use fresh hot pictures instead.  The API works well if you have a variety of pictures from different people, so as to provide more contrast when recognizing pictures later.

Also, remember that everything you do in the Test Console can be programmatically accessed using Mashape’s different client libraries.)

Here’s a response example when you upload a picture:

image

 

 

Once you’ve done this for all your pictures, it’s time to “Rebuild” them.

4. The “Rebuild Album” endpoint only requires your album and albumkey.  It will also make sure you have uploaded enough pictures and entries to make.  Hit “Test Endpoint” to rebuild the album.  A successful response would look like this:

image

 

Let’s now move on to the 2nd main section of this tutorial which is to recognize a picture based on the album we have created in Face Recognition.

 

Recognize a Face using the Face Recognition API

In this section we will refer to a web app we have put up in jsfiddle.net and explain how it works.  You can access it here.

image

 

(Screenshot of the Face Recognition sample code in jsfiddle.net)

The main parts of this section will delve on 1) taking a webcam picture using photobooth.js, and 2) uploading the picture to the Recognize endpoint.  Let’s lay out the source code here so we can easily refer to each line code in explaining the two steps above.

Line 1: Use jQuery to initialize stuff when web page is “ready”

Line 3:  We are using the photobooth.js library to access the webcam with HTML5.  This single line instantiates the photobooth object on the “#photo” element and sets up the event handler when the camera icon is clicked.  It will execute the code inside the anonymous function when the camera icon is clicked.

Line 4 and 12:  photobooth returns a “data url” that represents the image we took with the camera.  We need to convert this into a binary blob, a format fit for uploading.

Line 5 – 6:  We just want to make sure that we got an image, and size is not zero.

Line 7 – 28:  We will now upload the picture to the Recognize endpoint.

Line 29 – 33:  This sets up a FormData that would hold the query parameters needed by the Recognize endpoint.  This is similar to the parameters that you see in the Test Console for the Recognize endpoint.

Line 35 – 43:  We set up a jQuery “ajax” call for Recognize endpoint with the required parameters:

– URL (the Face Recognition API endpoint in Mashape), type (HTTP POST), data (the form data parameters), headers (Mashape auth headers).

Line 44 – 48:  When we get a response back from the endpoint, display it with an alert.

The rest of the code required for this application (HTML, CSS) can be viewed in jsfiddle here.

image

 

The application is straightforward;  you need to plug in your Mashape key, and two other parameters from Face Recognition – album and album key.  Whenever you take a picture, it gets uploaded using the Face Recognition Recognize endpoint, and we display the result back.

A response would look similar to this:

As you can see, the response includes coordinates for the eyes, nose, mouth, gender, smile, and recognition based on the pictures that were “trained”.  Obviously, the more pictures you have (of you and other people), the better the recognition.

We hope that this tutorial helps you understand the Face Recognition API better.  We’d like to see your own Face Recognition projects.  Send us an email at support@mashape.com!

 

Advertisements

Need APIs for your Windows 8 apps?

*UPDATE – You can check the video tutorial of this post here*

If you’re looking for API ingredients for your Windows 8 applications, go check out Mashape.  They are an API Marketplace for your apps.  Instead of Googling/Binging your way to find new APIs, Mashape consolidates them for you, ready to be consumed.  No lengthy documentation whatsoever.  (Well, we still have to pay attention to documentation sometimes).  At this point they have 1100+ APIs listed there.  That’s good news for you, especially if you’re in a rush to win a hackathon or something 🙂

For C#, they currently have documentation here – https://www.mashape.com/docs/consume/csharp.   However if you’re looking for an async/await sample apt for your Windows 8 app, you’re in luck.  I have posted a sample code in Github that uses HttpClient (REST) to access Mashape-hosted APIs.

Before you download and run it, you should know that –

  1. The sample uses Bitly and WordCloudMaker APIs from Mashape.
  2. For both services, you would need to get a Mashape Authentication Header.
  3. For Bitly, you need a developer login and legacy API key.
  4. For WordCloudMaker, no API key is needed.
  5. Once you have retrieved info for number 2 and 3, you can plug them in to fields at the very top of the MainPage.xaml.cs file

Once you’re done with the above, test by hitting F5.  Study the code and start copy-pasting! 🙂

If you have questions or want to share a cool Windows 8/Mashape app that you’ve built, email chris@mashape.com

Happy coding!

Want to test something on Surface RT?

Instead of doing a review of Microsoft’s Surface RT device (of which there are a lot anyway), I figured I’d just ask you guys what you want me to test on it.  Or if you have apps you want to test, I can check them out too.  Just post your requests in the comments section below and I’ll try my best to test them out.

Setting up Parse Cloud Code tool in Windows

*UPDATE 2012-11-01 – Parse has released an official Windows tool for Cloud Code.  Check it here*

Parse is a Backend-As-A-Service (BAAS) for your mobile apps.  It promotes a development model where you can worry less about the server (backend) side of your application and focus more on the client side (Javascript, iOS, Android).  I posted a video here that briefly explains what Parse is and how to get started.

They have recently rolled out a feature called Cloud Code which lets you build code on the Parse cloud and expose them as functions (APIs) that can be called in your client-side Parse code.  This allows for the same benefits that we enjoy in a traditional API model.

Below are the steps on how to get your Parse Cloud Code tool running on Windows – we’re using Cygwin.  (The original steps can be found in the Parse Cloud Code Guide page).

  1. Install Cygwin from this page.  Just click the setup.exe link on that page and go through normal installation process (keep clicking Next unless you know what you’re doing) until you get to the “Select Packages” window that allows you to pick the packages for your Cygwin installation (see image below to see what this looks like).  Cygwin allows you to run Unix-based tools in Windows.
Cygwin Select Packages window

Cygwin Select Packages window

2.  We’re now going to install the Python and curl packages.  We need Python because the Parse Cloud code tool was coded in Python.  We need curl because it is called in the installer script that we will run later.

Start by searching “Python” in the search field and navigating to the Python hierarchy below when the results come up.  (Note:  Be careful not to hit “Enter” after typing in Python.  It will search as you type.  You might accidentally start downloading if you hit Enter).  Click on the “Default” word beside it until it says “Install”.  Mine looks like this (see image below). (Frankly I’m not sure which specific Python to install so I installed the whole branch.  It worked for me, but it would be great if anyone can correct me here).

Select the Python branch and install

Select the Python branch and install

After Python, do the same thing with curl.  I clicked to “Install” on All (again because I’m not sure 😛 – appreciate if anyone can correct me here)

curl

curl

Once these packages are chosen, click “Next” to start the download.

If everything’s successful, you should have an application called “Cygwin Terminal”  If you run it, it would look similar to the image below.  This gives us a Unix environment where we can type in the commands in the Cloud Code guide.  Type “pwd” to show you your current directory.

Cygwin Terminal

Cygwin Terminal

3.  Download the installer script (installer.sh) by clicking here.  This installer script downloads the actual Parse Cloud Code tool, and puts it in the proper folder so we can call it from anywhere inside Cygwin.  Note that this script cannot be executed inside a DOS command prompt – it should be inside Cygwin, and in a folder accessible by the Cygwin terminal.

In my case, the “/home/Chris” folder from the image above maps to “C:\Users\Chris\Documents\cygwin\home\Chris” because I chose to install Cygwin inside the My Documents folder.  Normally, you’d install it in your root drive (e.g. C:\\).  In my case, I placed the installer.sh file in a “parse” folder I created inside “/home/Chris”.  See image below to see what this looks like.

Now that looks familiar

Now that looks familiar

4.  Before we run the installer.sh in Cygwin, we need to make a minor change.  Open up the installer.sh file with your favorite text editor (I use TextPad).  Replace “https” with “http”, without the ‘s’.  Save the file.

installer.sh - change https to http

installer.sh – change https to http

We’re doing this because you’ll run into a certificate error when you try to run the script.  I can’t be bothered fixing that part, so I’d rather be just a bit unsecure, with http 🙂

5.  Last step!  Go back to the Cygwin terminal and navigate to the folder where you placed installer.sh.  Once you’re there type “bash installer.sh”.  This will start a quick installation.  See mine below.

Run installer.sh then "parse"

Run installer.sh then “parse”

Type “parse” to check if the installation was successful.  If you get the parse parameters info, you’re good to go!  You can go back to the Parse Cloud Code guide and start from “Setting Up Cloud Code