AI for Earth Camera Trap Detection API

API change history

API for detecting animals, people, and vehicles in camera trap images using the MegaDetector model. This API is intended for real-time applications that process a small number of images at a time and require low latency; for batch processing applications, see the corresponding batch processing API.

Processes the input image(s) using the detection model.

Runs the detection model on up to eight images, optionally filtering detections based on a confidence threshold.

Request

Request URL

Request parameters

  • (optional)
    float

    The confidence threshold above which a proposed bounding box is considered a detection. Set it to a low value, such as 0.05, if you would like to receive all candidate boxes. When visualizing results using the render parameter, we recommend using a higher threshold, such as 0.8, for clearer visualizations.

  • (optional)
    boolean

    If true, the endpoint will return all input images annotated with detection bounding boxes with confidence above the confidence threshold, in addition to the json result.

Request headers

  • (optional)
    string
    Media type of the body sent to the API.
  • string
    Subscription key which provides access to this API. Found in your Profile.

Request body

Send up to 8 images to be processed as files in a multipart form.

  • The keys (image_name in the following example) in the files dictionary should be unique identifiers of the images, as the returned result will also be keyed by these.

  • Make sure to set the content media type correctly for each file, as only files of the accepted types will be processed.

  • The accepted image types are image/jpeg, image/png, application/octet-stream. Please send jpeg images of usual camera trap images size (500KB - 4MB).

For example, in Python:

  import requests
  import os

  num_images_to_upload = 3
  params = {
      'confidence': 0.8,
      'render': True
  }
  
  files = {}
  num_images = 0
  for i, image_name in enumerate(sorted(os.listdir(sample_input_dir))):
      if not image_name.lower().endswith('.jpg'):
          continue
      
      if num_images >= num_images_to_upload:
          break
      else:
          num_images += 1
      
      img_path = os.path.join(sample_input_dir, image_name)
      files[image_name] = (image_name, open(img_path, 'rb'), 'image/jpeg')
      
  r = requests.post(base_url + 'detect', params=params, files=files)

Responses

200 OK

Images are successfully processed. The detection bounding boxes, their confidences and categories will be returned in .json format. If the render parameter was true, annotated images will be returned as well. Since multiple objects of different types may be returned, the response is encoded using requests_toolbelt.multipart.encoder.MultipartEncoder.

A result is a json-encoded dictionary, where the keys are the unique image identifier that you specified for each image in the request body. Each value is an array containing the detections above confidence that were found on that image. The array is empty if no animals/people/vehicles are detected above the confidence threshold. Each detection is an array, formatted as [ymin, xmin, ymax, xmax, confidence, category], where the first four floats are the relative coordinates of the bounding box. category is one of 1 (animal), 2 (person), or 3 (vehicle).

Here is an example of how you can parse the result in Python:

import os
import json
from requests_toolbelt.multipart import decoder
from PIL import Image

results = decoder.MultipartDecoder.from_response(r)

text_results = {}
images = {}
for part in results.parts:
    # part is a BodyPart object with b'Content-Type', and b'Content-Disposition', the later includes 'name' and 'filename' info
    headers = {}
    for k, v in part.headers.items():
        headers[k.decode(part.encoding)] = v.decode(part.encoding)
    if headers.get('Content-Type', None) == 'image/jpeg':
        c = headers.get('Content-Disposition')
        image_name = c.split('name="')[1].split('"')[0]  # this is an HTTP string; you can parse it more elegantly using a library
        image = Image.open(io.BytesIO(part.content))
        
        images[image_name] = image
    
    elif headers.get('Content-Type', None) == 'application/json':
        text_result = json.loads(part.content.decode())
        
for img_name, img in sorted(images.items()):
  img.save(img_name + '.jpg')

text_result looks like this:

{
  'S1_D04_R6_PICT0022.JPG': [[0.011515299789607525,
   0.11399328708648682,
   0.9100480079650879,
   1.0,
   0.9953194260597229, 1]],
 'S1_D04_R6_PICT0128.JPG': [[0.5885017514228821,
   0.019416160881519318,
   0.6662894487380981,
   0.16861802339553833,
   0.8873217105865479, 2]],
 'S1_D04_R6_PICT0129.JPG': []
}

Representations

400 Bad Request

No image(s) of accepted types (image/jpeg, image/png, application/octet-stream) received, or the confidence parameter is not a float between 0 and 1.

413 Request Entity Too Large

More than 8 images are sent, or the total size of uploaded content is too big.

500 Internal Server Error

Error occurred reading the images, performing detection on the images, consolidating the results or drawing annotations (if requested). See the error message to diagnose the issue.

Code samples

@ECHO OFF

curl -v -X POST "https://aiforearth.azure-api.net/api/v1/camera-trap/sync/detect?confidence={float}&render=false"
-H "Content-Type: multipart/form-data"
-H "Ocp-Apim-Subscription-Key: {subscription key}"

--data-binary "{body}" 

using System;
using System.Net.Http.Headers;
using System.Text;
using System.Net.Http;
using System.Web;

namespace CSHttpClientSample
{
    static class Program
    {
        static void Main()
        {
            MakeRequest();
            Console.WriteLine("Hit ENTER to exit...");
            Console.ReadLine();
        }
        
        static async void MakeRequest()
        {
            var client = new HttpClient();
            var queryString = HttpUtility.ParseQueryString(string.Empty);

            // Request headers
            client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", "{subscription key}");

            // Request parameters
            queryString["confidence"] = "{float}";
            queryString["render"] = "false";
            var uri = "https://aiforearth.azure-api.net/api/v1/camera-trap/sync/detect?" + queryString;

            HttpResponseMessage response;

            // Request body
            byte[] byteData = Encoding.UTF8.GetBytes("{body}");

            using (var content = new ByteArrayContent(byteData))
            {
               content.Headers.ContentType = new MediaTypeHeaderValue("< your content type, i.e. application/json >");
               response = await client.PostAsync(uri, content);
            }

        }
    }
}	
// // This sample uses the Apache HTTP client from HTTP Components (http://hc.apache.org/httpcomponents-client-ga/)
import java.net.URI;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

public class JavaSample 
{
    public static void main(String[] args) 
    {
        HttpClient httpclient = HttpClients.createDefault();

        try
        {
            URIBuilder builder = new URIBuilder("https://aiforearth.azure-api.net/api/v1/camera-trap/sync/detect");

            builder.setParameter("confidence", "{float}");
            builder.setParameter("render", "false");

            URI uri = builder.build();
            HttpPost request = new HttpPost(uri);
            request.setHeader("Content-Type", "multipart/form-data");
            request.setHeader("Ocp-Apim-Subscription-Key", "{subscription key}");


            // Request body
            StringEntity reqEntity = new StringEntity("{body}");
            request.setEntity(reqEntity);

            HttpResponse response = httpclient.execute(request);
            HttpEntity entity = response.getEntity();

            if (entity != null) 
            {
                System.out.println(EntityUtils.toString(entity));
            }
        }
        catch (Exception e)
        {
            System.out.println(e.getMessage());
        }
    }
}


<!DOCTYPE html>
<html>
<head>
    <title>JSSample</title>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.0/jquery.min.js"></script>
</head>
<body>

<script type="text/javascript">
    $(function() {
        var params = {
            // Request parameters
            "confidence": "{float}",
            "render": "false",
        };
      
        $.ajax({
            url: "https://aiforearth.azure-api.net/api/v1/camera-trap/sync/detect?" + $.param(params),
            beforeSend: function(xhrObj){
                // Request headers
                xhrObj.setRequestHeader("Content-Type","multipart/form-data");
                xhrObj.setRequestHeader("Ocp-Apim-Subscription-Key","{subscription key}");
            },
            type: "POST",
            // Request body
            data: "{body}",
        })
        .done(function(data) {
            alert("success");
        })
        .fail(function() {
            alert("error");
        });
    });
</script>
</body>
</html>

#import <Foundation/Foundation.h>

int main(int argc, const char * argv[])
{
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    
    NSString* path = @"https://aiforearth.azure-api.net/api/v1/camera-trap/sync/detect";
    NSArray* array = @[
                         // Request parameters
                         @"entities=true",
                         @"confidence={float}",
                         @"render=false",
                      ];
    
    NSString* string = [array componentsJoinedByString:@"&"];
    path = [path stringByAppendingFormat:@"?%@", string];

    NSLog(@"%@", path);

    NSMutableURLRequest* _request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:path]];
    [_request setHTTPMethod:@"POST"];
    // Request headers
    [_request setValue:@"multipart/form-data" forHTTPHeaderField:@"Content-Type"];
    [_request setValue:@"{subscription key}" forHTTPHeaderField:@"Ocp-Apim-Subscription-Key"];
    // Request body
    [_request setHTTPBody:[@"{body}" dataUsingEncoding:NSUTF8StringEncoding]];
    
    NSURLResponse *response = nil;
    NSError *error = nil;
    NSData* _connectionData = [NSURLConnection sendSynchronousRequest:_request returningResponse:&response error:&error];

    if (nil != error)
    {
        NSLog(@"Error: %@", error);
    }
    else
    {
        NSError* error = nil;
        NSMutableDictionary* json = nil;
        NSString* dataString = [[NSString alloc] initWithData:_connectionData encoding:NSUTF8StringEncoding];
        NSLog(@"%@", dataString);
        
        if (nil != _connectionData)
        {
            json = [NSJSONSerialization JSONObjectWithData:_connectionData options:NSJSONReadingMutableContainers error:&error];
        }
        
        if (error || !json)
        {
            NSLog(@"Could not parse loaded json with error:%@", error);
        }
        
        NSLog(@"%@", json);
        _connectionData = nil;
    }
    
    [pool drain];

    return 0;
}

<?php
// This sample uses the Apache HTTP client from HTTP Components (http://hc.apache.org/httpcomponents-client-ga/)
require_once 'HTTP/Request2.php';

$request = new Http_Request2('https://aiforearth.azure-api.net/api/v1/camera-trap/sync/detect');
$url = $request->getUrl();

$headers = array(
    // Request headers
    'Content-Type' => 'multipart/form-data',
    'Ocp-Apim-Subscription-Key' => '{subscription key}',
);

$request->setHeader($headers);

$parameters = array(
    // Request parameters
    'confidence' => '{float}',
    'render' => 'false',
);

$url->setQueryVariables($parameters);

$request->setMethod(HTTP_Request2::METHOD_POST);

// Request body
$request->setBody("{body}");

try
{
    $response = $request->send();
    echo $response->getBody();
}
catch (HttpException $ex)
{
    echo $ex;
}

?>
########### Python 2.7 #############
import httplib, urllib, base64

headers = {
    # Request headers
    'Content-Type': 'multipart/form-data',
    'Ocp-Apim-Subscription-Key': '{subscription key}',
}

params = urllib.urlencode({
    # Request parameters
    'confidence': '{float}',
    'render': 'false',
})

try:
    conn = httplib.HTTPSConnection('aiforearth.azure-api.net')
    conn.request("POST", "/api/v1/camera-trap/sync/detect?%s" % params, "{body}", headers)
    response = conn.getresponse()
    data = response.read()
    print(data)
    conn.close()
except Exception as e:
    print("[Errno {0}] {1}".format(e.errno, e.strerror))

####################################

########### Python 3.2 #############
import http.client, urllib.request, urllib.parse, urllib.error, base64

headers = {
    # Request headers
    'Content-Type': 'multipart/form-data',
    'Ocp-Apim-Subscription-Key': '{subscription key}',
}

params = urllib.parse.urlencode({
    # Request parameters
    'confidence': '{float}',
    'render': 'false',
})

try:
    conn = http.client.HTTPSConnection('aiforearth.azure-api.net')
    conn.request("POST", "/api/v1/camera-trap/sync/detect?%s" % params, "{body}", headers)
    response = conn.getresponse()
    data = response.read()
    print(data)
    conn.close()
except Exception as e:
    print("[Errno {0}] {1}".format(e.errno, e.strerror))

####################################
require 'net/http'

uri = URI('https://aiforearth.azure-api.net/api/v1/camera-trap/sync/detect')

query = URI.encode_www_form({
    # Request parameters
    'confidence' => '{float}',
    'render' => 'false'
})
if query.length > 0
  if uri.query && uri.query.length > 0
    uri.query += '&' + query
  else
    uri.query = query
  end
end

request = Net::HTTP::Post.new(uri.request_uri)
# Request headers
request['Content-Type'] = 'multipart/form-data'
# Request headers
request['Ocp-Apim-Subscription-Key'] = '{subscription key}'
# Request body
request.body = "{body}"

response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
    http.request(request)
end

puts response.body