bigml.io - the bigml api

BigML.io: The BigML API

October 12, 2012

BigML Inc BigML.io: The BigML API October 12, 2012 1 / 66

1 Introduction

2 BigML Resources

3 Sources

4 Datasets

5 Models

6 Predictions

7 Evaluations

8 Bindings

9 Final Remarks

BigML.io: Base URL

https://bigml.io

Base URL

A RESTful API for creating and managing BigML resourcesprogrammatically.

All accesses are performed over HTTPS.

BigML.io: Development Mode

https://bigml.io/dev/

Dev Mode

No credits are charged.

Limited to 1MB per resource but unlimited in the number of resources.

BigML.io: Version

https://bigml.io/andromeda/

Version

BigML.io first version is named andromeda.

If you omit the version name in your API requests, you will get access tothe latest API version.

BigML.io: Authentication

1 BIGML_USERNAME=alfred

2 BIGML_API_KEY=62270d2ad14eba4e349432e80d749342de5550a4

3 BIGML_AUTH="username=$BIGML_USERNAME;api_key=$BIGML_API_KEY"

Authentication

All accesses to BigML.io need to be authenticated.

Authentication is performed including your username and your BigML APIKey in every request.

If you use an environment variable (e.g. BIGML AUTH) you can keep yourcredentials out of your source code.

BigML Resources

Source Dataset Model Prediction

A source is a filecontaining theraw data that

you want to useto create apredictive

A dataset is astructured

version of adata sourcewhere eachcolumn has

been assigned atype

A model iscreated using a

dataset asinput, selectingwhich fields to

use as inputand which field

will be theobjective

A prediction iscreated using amodel and thenew instance

that you want toclassify as input

BigML.io: Source

sepal length,sepal width,petal length,petal width,species

5.1,3.5,1.4,0.2,Iris-setosa

7.0,3.2,4.7,1.4,Iris-versicolor

5.8,2.7,5.1,1.9,Iris-virginica

Create a New Source

A source is the raw data that you want to use to create a predictivemodel.

A source is usually a (big) file in tabular format.

Each column in the file represents a feature or field.

By default, the last column represents the class or objective field.

The file may have a first row or header with a name for each field.

BigML.io: Source

https://bigml.io/source

Source Base URL

Datasources can be created using several data sources:Local filesRemote data accessed via HTTP or HTTPsFiles in S3 bucketsBlobs in Windows Azure storageInline data contained in the datasource creation request

Data must be in tabular format, cannot be bigger than 64GB, andcan be compressed (.Z or .gz, but not .zip)

BigML.io: Creating a Source using a local file

curl https://bigml.io/source?$BIGML_AUTH -F file=@iris.csv

Creating a Source

The file must be attached in the post as a file upload

The Content-Type in your HTTP request must bemultipart/form-data, as specified by RFC2388.

BigML.io: Creating a Source using a remote URL

curl https://bigml.io/source?$BIGML_AUTH \

-X "POST" \

-H "content-type: application/json" \

-d '{"remote": "https://static.bigml.com/csv/iris.csv"}'

Creating a Remote Source

The Content-Type in your HTTP request must be application/json.

URLs can be HTTP or HTTPS with realm authentication, public orprivate Amazon S3, or Windows Azure files.

BigML.io: Creating a Source using inline data

-X "POST" \

-d '{"data": "a,b,c,d\n1,2,3,4\n5,6,7,8"}'

Creating an Inline Source

The Content-Type in your HTTP request must be application/json.

Source data is included in the JSON body as a string with key“data”.

Maximum size of inline sources is 10MB.

BigML.io: New Source

2 "category": 0,

3 "code": 201,

4 "content_type": "application/octet-stream",

5 "created": "2012-05-21T18:41:47.546669",

6 "credits": 0.0,

7 "description": "",

8 "file_name": "iris.csv",

9 "md5": "d1175c032e1042bec7f974c91e4a65ae",

10 "name": "iris.csv",

11 "number_of_datasets": 0,

12 "number_of_models": 0,

13 "number_of_predictions": 0,

14 "private": true,

15 "resource": "source/4f52824203ce893c0a000053",

16 "size": 4608,

17 "source_parser": {},

18 "status": {

19 "code": 2,

20 "elapsed": 0,

21 "message": "The source creation has been started"

23 "tags": [],

24 "type": 0,

25 "updated": "2012-05-21T18:41:47.546693"

New Source

BigML.io: Source Arguments

One Required Type Description

file multipart form data File.remote String URL of the remote source.data String Inline data in tabular format.

Optional Type Description

category Integer The category that best describes the data.description String A description of the source of up to 8192 characters.name String The name you want to give to the new source.private Boolean Whether you want your source to be private or not.source parser Object Set of parameters to parse the source.tags List A list of strings that help classify and index your source.

Table : Source Arguments

BigML.io: Creating a Source with args

-X "POST" \

-d '{"remote": "https://static.bigml.com/csv/iris.csv", "name": "iris"}'

Creating a Source with args

BigML.io: Source Parser

2 "header": true,

3 "locale": "en-US",

4 "missing_tokens": ["?"],

5 "quote": "\"",

6 "separator": ",",

7 "trim": true

Source Parser

BigML.io: Updating a Source

curl https://bigml.io/source/4f64191d03ce89860a000000?$BIGML_AUTH \

-X PUT \

-H 'content-type: application/json' \

-d '{"name": "a new name", "source_parser": {"locale": "es-ES"}}'

Updating a Source

BigML.io: Deleting a Source

curl "https://bigml.io/source/4f603fe203ce89bb2d000000?$BIGML_AUTH" \

-X DELETE

Deleting a Source

Response HTTP/1.1 204 NO CONTENT

BigML.io: Retrieving a Source

curl \

"https://bigml.io/source/4eee50b90a590f7d5c000008?$BIGML_AUTH"

Retrieving a Source via BigML.io

https://bigml.com/dashboard/source/4eee50b90a590f7d5c000008

Visualizing a Source via BigML.com

BigML.io: Source Properties

property type filterable sortable updatable

category Integer yes yes yescode Integer no no nocontent type String yes yos nocreated Datetime yes yes nocredits Float yes yes nodescription String yes yes yesfields Object no no nofile name String yes yes nomd5 String no no noname String yes yes yesnumber of datasets Integer yes yes nonumber of models Integer yes yes nonumber of predictions Integer yes yes noprivate Boolean yes yes yesresource String no no norows Integer yes yes nosize Integer yes yes nosource String yes yes nosource status String yes yes nostatus Object no no notags List yes yes yesupdated Datetime yes yes no

Table : Source Properties

BigML.io: Listing Sources

curl "https://bigml.io/source?limit=10;offset=10;$BIGML_AUTH"

Listing Sources

limit Specifies the number of sources to retrieve. Must be lessthan or equal to 200.

offset The position of the whole source list at which the retrievedsource list will start off.

BigML.io: Listing Sources (cont.)

2 "meta": {

3 "limit": 10,

4 "next": "/source?limit=10&offset=20&username=francisco&api_key=aa4420adaed03ea68c850",

5 "offset": 10,

6 "previous": null,

7 "total_count": 540

9 "objects": [

11 "code": 200,

12 "content_type": "text/csv",

13 ...

15 ...

17 "code": 200,

18 "content_type": "text/csv",

19 ...

Source Listing

BigML.io: Filtering Sources

curl "https://bigml.io/source?size_gt=1048576;$BIGML_AUTH"

Retrieving sources bigger than 1 MB

Filter Description

lt Less thanlte Less than or equal togt Greater thangte Greater than or equal to

Table : Filtering Arguments

BigML.io: Sorting Sources

curl "https://bigml.io/source?order_by=-size;$BIGML_AUTH"

Sorting sources by size

order by Specifies the order of the sources to retrieve. Must be oneof the sortable fields. If you prefix the field name with “-”,the order will be descending.

BigML.io: Dataset

https://bigml.io/dataset

Dataset Base URL

A dataset is a structured version of a source where each field hasbeen processed and serialized according to its type.

A field can be numeric or categorical.

Datetime and text fields are coming down the pike.

BigML.io: Create a New Dataset

curl "https://bigml.io/andromeda/dataset?$BIGML_AUTH" \

-X POST \

-d '{"source": "/source/4ee5761c80e1c664f1000000"}'

Create a New Dataset

1 { "category": 0,

2 "code": 201,

3 "columns": 5,

4 "created": "2012-05-25T06:02:40.889538",

5 "credits": 0.0087890625,

7 "fields": {

8 "000000": {

9 "column_number": 0,

10 "name": "sepal length",

11 "optype": "numeric"

13 ...

15 "locale": "en_US",

16 "name": "iris' dataset",

17 "number_of_models": 0,

19 "private": true,

20 "resource": "dataset/4f66a0b903ce8940c5000000",

21 "rows": 0,

22 "size": 4608,

23 "source": "source/4f665b8103ce8920bb000006",

24 "source_status": true,

25 "status": {

26 "code": 1,

27 "message": "The dataset is being processed and will be created soon"

29 "tags": [],

30 "updated": "2012-05-25T06:02:40.889570" }

New Datatset

BigML.io: Dataset Arguments

Required Type Description

source String Valid source/id

category Integer The category that best describes the dataset.description String A description of the dataset of up to 8192 characters.fields Object The fields that you want to use to create the dataset.name String Name of the dataset.private Boolean Whether you want your dataset to be private or not.size Integer Maximum number of bytes to process.tags List A list of strings that help classify and index your dataset.

Table : Dataset Arguments

BigML.io: Creating a Dataset with args

curl "https://bigml.io/dataset?$BIGML_AUTH" \

-X POST \

-d '{"source": "/source/4ee5761c80e1c664f1000000", "name": "my dataset"}'

Creating a Dataset with args

BigML.io: Updating a Dataset

curl https://bigml.io/dataset/4f66a0b903ce8940c5000000?$BIGML_AUTH \

-X PUT

-d '{"name": "a new name"}'

Updating a Dataset

BigML.io: Deleting a Dataset

curl "https://bigml.io/dataset/4f66a0b903ce8940c5000000?$BIGML_AUTH" \

-X DELETE

Deleting a Dataset

BigML.io: Retrieving a Dataset

curl "https://bigml.io/dataset/4f66a0b903ce8940c5000000?$BIGML_AUTH"

Retrieving a Dataset via BigML.io

https://bigml.com/dashboard/dataset/4f66a0b903ce8940c5000000

Retrieving a Dataset via BigML.com

BigML.io: Dataset Properties

category Integer yes yes yescode Integer no no nocolumns Integer yes yes nocreated Datetime yes yes nocredits Float yes yes nodescription String yes yes yesfields Object no no nolocale String no no noname String yes yes yesnumber of models Integer yes yes nonumber of predictions Integer yes yes noprivate Boolean yes yes yesresource String no no norows Integer yes yes nosize Integer yes yes nosource String yes yes nosource status Boolean yes yes nostatus Object no no notags List yes yes yesupdated Datetime yes yes no

Table : Dataset Properties

BigML.io: Listing Datasets

curl "https://bigml.io/dataset?limit=10;offset=10;$BIGML_AUTH"

Listing Datasets

limit The total number of datasets to retrieve (≤ 200).

offset The offset at which the dataset listing will start.

BigML.io: Dataset Listing

2 "meta": {

3 "limit": 10,

4 "next": "/dataset?limit=10&offset=20&username=ciskoo&api_key=70aaae8d77699bc5d43788a5",

5 "offset": 10,

6 "previous": null,

9 "objects": [

11 "code": 200,

12 "columns": 120,

13 ...

15 ...

17 "code": 200,

18 "columns": 5,

19 ...

Dataset Listing

BigML.io: Filtering Datasets

curl "https://bigml.io/dataset?size_gt=1048576;$BIGML_AUTH"

Retrieving datasets bigger than 1 MB

Filter Description

BigML.io: Sorting Datasets

curl "https://bigml.io/dataset?order_by=-size;$BIGML_AUTH"

Sorting datasets by size

order by Specifies the order of the datasets to retrieve. Must be oneof the sortable fields. If you prefix the field name with “-”,they will be given in descending order.

BigML.io: Model

https://bigml.io/model

Model Base URL

A model is a tree-like representation of your dataset withpredictive power.

You can create a model selecting which fields from your datasetyou want to use as input fields (or predictors) and which field youwant to predict, the objective field.

BigML.io: Create a New Model

curl https://bigml.io/model?$BIGML_AUTH \

-X POST \

-d '{"dataset": "dataset/4f66a80803ce8940c5000006"}'

Create a New Model

1 { "category": 0,

2 "code": 201,

3 "columns": 5,

4 "created": "2012-05-25T07:13:07.243623",

5 "credits": 0.03515625,

6 "dataset": "dataset/4f66a80803ce8940c5000006",

7 "dataset_status": true,

9 "holdout": 0.0,

10 "input_fields": [],

12 "max_columns": 5,

13 "max_rows": 150,

14 "name": "iris' dataset model",

16 "objective_fields": [],

17 "private": true,

18 "range": [

19 1, 150

21 "resource": "model/4f67c0ee03ce89c74a000006",

22 "rows": 150,

23 "size": 4608,

26 "status": {

27 "code": 1, "message": "The model is being processed and will be created soon"

29 "tags": [],

30 "updated": "2012-05-25T07:13:07.243658" }

New Model

BigML.io: Model Arguments

dataset String Valid dataset/id

category Integer The category that best describes the dataset.description String A description of the dataset of up to 8192 characters.input fields List The fields that you want to use to create the model.name String Name of the dataset.objective fields List The field that you want to predict.private Boolean Whether you want your dataset to be private or not.range List The range of successive instances to build the model.tags List A list of strings that help classify your dataset.

Table : Model Arguments

BigML.io: Creating a Model with args

curl https://bigml.io/andromeda/model?$BIGML_AUTH \

-X POST \

-d '{"dataset": "dataset/4f66a80803ce8940c5000006", "input_fields": ["000001", "000003"]}'

Creating a Model with args

BigML.io: Updating a Model

curl https://bigml.io/model/4f67c0ee03ce89c74a000006?$BIGML_AUTH \

-X PUT \

Updating a Model

BigML.io: Deleting a Model

curl "https://bigml.io/model/4f67c0ee03ce89c74a000006?$BIGML_AUTH"

-X DELETE

Deleting a Model

BigML.io: Retrieving a Model

curl "https://bigml.io/model/4f66a80803ce8940c5000006?$BIGML_AUTH"

Retrieving a Model via BigML.io

https://bigml.com/dashboard/model/4f66a80803ce8940c5000006

Retrieving a Model via BigML.com

BigML.io: Model Properties

category Integer yes yes yescode Integer no no nocolumns Integer yes yes nocreated Datetime yes yes nocredits Float yes yes nodataset String yes yes nodataset status Boolean yes yes nodescription String yes yes yesinput fields Object no no nolocale String no no nomax columns Integer yes yes nomax rows Integer yes yes nomodel Object no no noname String yes yes yesnumber of predictions Integer yes yes noobjective fields List no no noprivate Boolean yes yes yesrange List no no noresource String no no nosize Integer yes yes nostatistical pruning Boolean yes yes nostatus Object no no notags List yes yes yesupdated Datetime yes yes no

Table : Model Properties

BigML.io: Listing Models

curl "https://bigml.io/model?limit=10;offset=10;$BIGML_AUTH"

Listing Models

limit The number of models to retrieve (≤ 200).

offset The offset at which the model listing will start off.

BigML.io: Model Listing

2 "meta": {

3 "limit": 10,

4 "next": "/model?limit=10&offset=20&username=ciskoo&api_key=70aaae8d77699bc5d437876d85",

5 "offset": 10,

6 "previous": null,

9 "objects": [

11 "code": 200,

12 "columns": 1150,

13 ...

15 ...

17 "code": 200,

18 "columns": 512,

19 ...

Model Listing

BigML.io: Filtering Models

curl "https://bigml.io/model?size_gt=1048576;$BIGML_AUTH"

Retrieving models bigger than 1 MB

Filter Description

BigML.io: Sorting Models

curl "https://bigml.io/model?order_by=-size;$BIGML_AUTH"

Sorting models by size

order by Specifies the order of the models to retrieve. Must be oneof the sortable fields. If you prefix the field name with “-”,they will be given in descending order.

BigML.io: Prediction

https://bigml.io/prediction

Prediction Base URL

A prediction is created using a model/id and the properties of thenew instance (input data) for which you wish to create a prediction.

To create a new prediction, BigML.io will automatically navigate thecorresponding model to find the leaf node that best classifies thenew instance.

BigML.io: Create a New Prediction

curl https://bigml.io/prediction?$BIGML_AUTH \

-X POST \

-d '{"model": "model/4f67c0ee03ce89c74a000006",

"input_data": {"000001": 3}}'

Create a New Prediction

1 { "code": 201,

2 "created": "2012-03-21T16:26:51.300678",

3 "credits": 0.01,

6 "fields": { ... },

7 "input_data": { "000001": 3 },

8 "locale": "en-US",

9 "model": "model/4f67c0ee03ce89c74a000006",

10 "model_status": true,

11 "name": "Prediction for species",

12 "objective_fields": [ "000004" ],

13 "prediction": { "000004": "Iris-virginica" },

14 "prediction_path": {

15 "bad_fields": [],

16 "next_predicates": [

17 { "count": 100, "field": "000002", "operator": ">", "value": 2.45 },

18 { "count": 50, "field": "000002", "operator": "<=", "value": 2.45 }

20 "path": [],

21 "unknown_fields": []

23 "private": true,

24 "resource": "prediction/4f6a014b03ce89584500000f",

27 "status": { "code": 5, "message": "The prediction has been created" },

28 "updated": "2012-03-21T16:26:51.300700" }

New Prediction

BigML.io: Prediction Arguments

model String Valid model/id.input data Object Field’s id/value pairs representing the instance.

category Integer The category that best describes the dataset.description String A description of the dataset of up to 8192 characters.name String Name of the dataset.private Boolean Whether you want your dataset to be private or not.tags List A list of strings that help classify and index your dataset.

Table : Prediction Arguments

BigML.io: Creating a Prediction with args

curl https://bigml.io/andromeda/prediction?$BIGML_AUTH \

-X POST \

-d '{"input_data": {"000001": 3},

"model": "model/4f67c0ee03ce89c74a000006",

"name": "my prediction"}'

Creating a Prediction with args

BigML.io: Updating a Prediction

curl https://bigml.io/prediction/4f6a014b03ce89584500000f?$BIGML_AUTH \

-X PUT \

Updating a Prediction

BigML.io: Deleting a Prediction

curl "https://bigml.io/prediction/4f6a014b03ce89584500000f?$BIGML_AUTH"

-X DELETE

Deleting a Prediction

BigML.io: Retrieving a Prediction

curl "https://bigml.io/prediction/4f6a014b03ce89584500000f?$BIGML_AUTH"

Retrieving a Prediction via BigML.io

https://bigml.com/dashboard/prediction/4f6a014b03ce89584500000f

Retrieving a Prediction via BigML.com

BigML.io: Prediction Properties

category Integer yes yes yescode Integer no no nocreated Datetime yes yes nocredits Float yes yes nodataset String yes yes nodataset status Boolean yes yes nodescription String yes yes yesfields Object no no noinput data Object no no nolocale String no no nomodel String yes yes nomodel status Boolean yes yes noname String yes yes yesobjective fields List yes yes noprediction Object yes yes noprediction path Object no no noprivate Boolean yes yes yesresource String no no nosource String yes yes nosource status Boolean yes yes nostatus Object no no notags List yes yes yesupdated Datetime yes yes no

Table : Prediction Properties

BigML.io: Listing Predictions

curl "https://bigml.io/prediction?limit=10;offset=10;$BIGML_AUTH"

Listing Predictions

limit The number of predictions to retrieve (≤ 200).

offset The offset at which the prediction listing will start off.

1 { "category": 0,

2 "code": 201,

3 "created": "2012-05-25T07:20:35.687797",

4 "credits": 0.01,

8 "fields": { ... },

9 "input_data": { "000001": 3 },

11 "model": "model/4f67c0ee03ce89c74a000006",

12 "model_status": true,

13 "name": "Prediction for species",

14 "objective_fields": [ "000004" ],

15 "prediction": { "000004": "Iris-virginica" },

16 "prediction_path": {

17 "bad_fields": [],

18 "next_predicates": [

19 { "field": "000002", "operator": ">", "value": 2.45 },

20 { "field": "000002", "operator": "<=", "value": 2.45 }

22 "path": [], "unknown_fields": [] },

23 "private": true,

24 "resource": "prediction/4f6a014b03ce89584500000f",

27 "status": { "code": 5, "message": "The prediction has been created" },

28 "tags": [],

29 "updated": "2012-05-25T07:20:35.687819" }

Prediction Listing

BigML.io: Filtering Predictions

curl "https://bigml.io/prediction?created__gt=2012-01-12;$BIGML_AUTH"

Retrieving predictions created after 12/1/2012

Filter Description

BigML.io: Sorting Predictions

curl "https://bigml.io/prediction?order_by=-name;$BIGML_AUTH"

Sorting predictions by name

order by Specifies the order of the predictions to retrieve. Must beone of the sortable fields. If you prefix the field name with“-”, they will be given in descending order.

BigML.io: Evaluation

https://bigml.io/evaluation

Evaluation Base URL

An evaluation automatically measures the performance of a modelcorrectly predicting the objective field for a pre-labeled test set.

An evaluation is created using the model/id of the model underevaluation and the a dataset/id of the testset.

BigML.io: Public Bindings

Bash https://github.com/bigmlcom/bigml-bash

Python https://github.com/bigmlcom/python

R https://github.com/bigmlcom/bigml-r

iOS https://github.com/fgarcialainez/ML4iOS

Java https://github.com/javinp/bigml-java

Ruby http://vigosan.github.com/big ml/

BigML.io: Final Remarks

dev mode Remember to include /dev in your URL requests to avoidcredit charges.

version Remember to include the current version name/andromeda in your URL requests to make sure thatfuture versions of the BigML API do not interfere with yourapplication.

bigml.io - the bigml api

Technology

wso2 api manager : going beyond the just api management

api world 2013 - transforming the netflix api

bigml api webinar - march 2014

emerging trends in the api economy and api management

bigml - oc.sti2.at · bigml goal highly scalable,...

(12) (10) patent no.: us 9,501,540 b2 united states...

bigml spring 2016 release

bigml summer 2016 release

merchant warrior - api documentation warrior api...direct...

html5 api - the screen orientation api and fullscreen

application programming interface (api) reference...

welcome to the api economy: developing your api strategy

bigml winter 2015 webinar

amazon lightsail - api reference · 2016-11-28 · amazon...

customizing the api store & publisher in wso2 api manager

package ‘bigml’ - the comprehensive r archive...

bigml spring 2014 webinar - clustering!

bigml project

amazon api gateway - api reference - docs.aws.amazon.com ·...

bigml winter 2017 release