FarmMaps/Documentatie
8
0
Fork 0
Code Pull Requests Releases Activity

Add Running tasks article and various corrections

Browse Source
This commit is contained in:
Auke Sytsma 2020-04-25 17:13:54 +02:00
parent 9a289b3683
commit 01db9fa14f
4 changed files with 486 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

215
Create-a-cropfield.md Normal file
View File

@ -0,0 +1,215 @@
## Creating a cropfield
This page explains what a cropfield is, and how to create one through the API.
Agricultural datasets generally refer to a specif crop, at a specific location at a specific time.
Therefore, we need to create a "Cropfield" item, to define this location and a timeframe our data relates to.
A cropfield provides a convienient way to group all data for a crop grown on a specific plot during a season.
Farmmaps also uses the cropfield to collect and prepare context data such as weather, satelite imagery etc.
To create a cropfield we:
* Get the parent folder to place the cropfield under
* Create a folder under this element (optional)
* Create the cropfield
**Prerequisites**
To create a cropfield we need:
* an acces token
* coordinates of the cropfield (what points define the plot contour)
* a startdate and an end date (what timeframe does this data aply to)
### Get parent element
First, we'll get the parent code needed to place the cropfield in the hierarchy.
**Request**
Replace `<acces token>` with your actual token.
```http
GET https://farmmaps.awacc.nl/api/v1/folders/my_drive? HTTP/1.1
Host: farmmaps.awacc.nl
Accept: application/json
Authorization: Bearer <access token>
```
**Response**
The response will be something similar to:
```http
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Tue, 21 Apr 2020 09:57:07 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: no-store,no-cache
Pragma: no-cache
Content-Encoding: br
Vary: Accept-Encoding
{
"url":"/api/v1/folders/f25d8765a1cd407cb235961c73c268cf:USER_FILES",
"code":"f25d8765a1cd407cb235961c73c268cf:USER_FILES",
"name":"My Drive",
"created":"2019-09-25T19:39:33.841835",
"updated":"2019-09-25T19:39:33.841835",
"itemType":"ROOT_FOLDER",
"size":0,
"state":0,
"thumbnail":false
}
```
So the `parentcode` we need is **"f25d8765a1cd407cb235961c73c268cf:USER_FILES"**
You can also find the code of a folder by browsing the folders in your FarmMaps account through the web interface.
The item code is the last part of the URL.
### Create folder
Depending on how you want to organise things, you might choose to make a separate folder for the cropfield.
A folder can be created by creating an item with itemType `FOLDER`.
> **Note:** At the moment, subfolders can not be created directly under the root folder, create a folder through the web interface first.
**Request**
```http
POST /api/v1/items HTTP/1.1
Accept: application/json
Authorization: Bearer <access token>
Content-Type: application/json
Cache-Control: no-cache
Host: farmmaps.awacc.nl
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 210
{
"parentCode": "f25d8765a1cd407cb235961c73c268cf:USER_FILES",
"itemType": "FOLDER",
"name": "My_new_folder",
"data": {},
"dataDate": "2020-04-24T09:59:18.893Z",
"geometry": {},
"tags": [ "string" ]
}
```
When the folder is created successfully we should recieve something like this:
**Response**
```http
{
"parentCode": "string",
"geometry": {},
"data": {},
"tags": [
"string"
],
"url": "string",
"code": "string",
"name": "My_new_folder",
"created": "2019-12-18T10:16:21.455Z",
"updated": "2019-12-18T10:16:21.455Z",
"dataDate": "2019-12-18T10:16:21.455Z",
"itemType": "string",
"sourceTask": "string",
"size": 0,
"state": 0,
"thumbnail": true
}
```
### Create the cropfield
Now we can create the cropfield. We do so by setting `"itemType": "vnd.farmmaps.itemtype.cropfield"` and adding data to the geometry parameter.
Generally, a cropfield contour would be defined as a polygon, so we add `"type":"Polygon" and all the coordinates.
We also need to specify a `dataDate` and a `dataEndDate` to indicate the the timeframe of the growing season.
The `dataDate` needs to be before `dataEndDate` and cannot be on the same day.
**Request**
```http
POST /api/v1/items HTTP/1.1
Accept: application/json
Authorization: Bearer <acces token>
Content-Type: application/json
Cache-Control: no-cache
Host: farmmaps.awacc.nl
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 916
{
"parentCode": "6601a06d812b40f9830c9fe4e63b1944",
"itemType": "vnd.farmmaps.itemtype.cropfield",
"name": "cropfield for VRA",
"dataDate": "2019-1-18T10:16:21.455Z",
"dataEndDate": "2019-12-18T10:16:21.455Z",
"data": {},
"geometry": {"type":"Polygon","coordinates":[[[6.09942873984307,53.070025028087],[6.09992507404607,53.0705617890585],[6.10036959220086,53.0710679529031],[6.10065149010421,53.0714062774307],[6.10087493644271,53.0716712354474],[6.10091082982487,53.0716936039203],[6.10165087441291,53.0712041549161],[6.10204994718318,53.0709349338005],[6.10263143118855,53.0705789370018],[6.10311578125011,53.0702657538294],[6.10331686552072,53.0701314102389],[6.103326530575,53.070119463569],[6.10309137950343,53.0699829669055],[6.10184241586523,53.0692902201371],[6.10168497998891,53.0691984306747],[6.10092987659869,53.0694894453514],[6.09942873984307,53.070025028087]]]}
}
```
**Response**
When the cropfield is created, it should be visible through the FarmMaps web interface (under "My Drive" in the respective parent folder).
We should have the following response:
```http
HTTP/1.1 201 Created
Server: nginx/1.14.0 (Ubuntu)
Date: Fri, 24 Apr 2020 08:56:45 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: no-store,no-cache
Pragma: no-cache
Content-Encoding: br
Location: /api/v1/items/8ecbaa2d85d5484db7f16b281b7cd013
Vary: Accept-Encoding
{
"parentCode":"6601a06d812b40f9830c9fe4e63b1944",
"geometry": {
"type":"Polygon",
"coordinates":[[
[6.09942873984307,53.070025028087],
[6.09992507404607,53.0705617890585],
[6.10036959220086,53.0710679529031],
[6.10065149010421,53.0714062774307],
[6.10087493644271,53.0716712354474],
[6.10091082982487,53.0716936039203],
[6.10165087441291,53.0712041549161],
[6.10204994718318,53.0709349338005],
[6.10263143118855,53.0705789370018],
[6.10311578125011,53.0702657538294],
[6.10331686552072,53.0701314102389],
[6.103326530575,53.070119463569],
[6.10309137950343,53.0699829669055],
[6.10184241586523,53.0692902201371],
[6.10168497998891,53.0691984306747],
[6.10092987659869,53.0694894453514],
[6.09942873984307,53.070025028087]]
]},
"data":{},
"tags":[],
"isEditable":false,
"url":"/api/v1/items/8ecbaa2d85d5484db7f16b281b7cd013",
"code":"8ecbaa2d85d5484db7f16b281b7cd013",
"name":"cropfield for VRA",
"created":"2020-04-24T08:56:45.3727807Z",
"updated":"2020-04-24T08:56:45.3727807Z",
"dataDate":"2019-01-18T10:16:21.455Z",
"itemType":"vnd.farmmaps.itemtype.cropfield",
"size":0,
"state":0,
"thumbnail":false
}
```
**Troubleshooting**
|Status code|Description|
|---|---|
|201|Cropfield created successfully|
|401|Error: not authenticated|
|403|Error: No write permissions in parent item|
|404|Error: Parent item not found|
Now that the cropfield has been created, we can start the processing by running a task.
This will collext all context data for the cropfield.
We need the ItemCode of the cropfield to start the task so keep this at hand.

6
Home.md
View File

@ -58,14 +58,14 @@ For testing purposes, [Postman](https://www.postman.com) can be used to perform
We provide the following guides:
* [Uploading a file](/wiki/Upload-a-file)
* Creating a cropfield
* Running a task
* [Creating a cropfield](/wiki/Create-a-cropfield)
* [Running a task](/wiki/Running-tasks)
Generally, tasks can be run in the same way.
However, each specific task has it's own inputs and properties.
How these work can be found in the use task examples below.
### Taks examples
### <a name="task-examples"></a>Task examples
* [VRAPoten-API](/wiki/VRAPoten-API)
* [VRANbs-API](/wiki/VRANbs-API)
* [VRAHerbicide-API](/wiki/VRAHerbicide-API)

265
Running-tasks.md Normal file
View File

@ -0,0 +1,265 @@
# Running a task
This page documents how to run a task from the FarmMaps API.
Once you have uploaded some data and made sure the relevant cropfield is available, you can now instruct the system to start processing the data.
Processing data is done by running tasks. There are many different types of tasks to acomplish many different things such as:
* Identifying the type of file to start automatic processing
* Converting a CSV file to a Shapefile
* Filtering data for valid values
* Creating a taskmap of input data.
## Workflow task.
There is one special type of task, the `workflow` task.
When an item (i.e. file-upload, folder, cropfield etc.) is created there are certain processing tasks that can be done by "default".
As an example, for an uploaded file these tasks are:
- Re-assembling the part-chunks back into a single file
- Identifying the type of file, and how it should be processed.
For a cropfield there are slightly different tasks such as:
- retrieve AHN mapping data (cut-out from public height maps)
- retrieve soil compaction data (based on a public map)
- retrieve shadow mapping data (based on satellite imagery)
These steps are all defined in the workflow for each `itemType`.
So, once an item has been created, we can run the `workflow` task to execute the predefined tasks automatically.
### Example: Running the workflow task for a cropfield.
We run a task by adding the `taskType` to the body of the request, with `vnd.farmmaps.task.workflow` as the type of task.
The item that the task will be executed on is specified in the url by it's itemcode. Other types of tasks can be run by changing the taskType.
**Request**
```http
POST /api/v1/items/98a480ad8d7444a8a10ef547cd8594eb/tasks HTTP/1.1
Authorization: Bearer <access token>
Accept: application/json
Content-Type: application/json
Cache-Control: no-cache
Host: farmmaps.awacc.nl
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 49
{
"taskType": "vnd.farmmaps.task.workflow"
}
```
**Response**
```http
HTTP/1.1 201 Created
Server: nginx/1.14.0 (Ubuntu)
Date: Sat, 25 Apr 2020 13:55:28 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: no-store,no-cache
Pragma: no-cache
Content-Encoding: br
Location: /api/v1/items/98a480ad8d7444a8a10ef547cd8594eb/tasks/52173fb5bf1146efa276ecfe1fda5f60
Vary: Accept-Encoding
{
"code":"52173fb5bf1146efa276ecfe1fda5f60",
"taskType":"vnd.farmmaps.task.workflow",
"attributes":{}
}
```
**Troubleshooting**
|Status code|Description|
|---|---|
|201|Task created|
|400|Error: Tasktype not found|
|401|Error: Not authenticated|
|403|Error: No WRITE permissions in item|
|404|Error: Item not found|
## Polling for task status
Now that the task was created, we can retrieve its status by performing a **GET** request on the same endpoint as used for starting task.
Note that the response contains a lot of tasks, and also multiple workflow tasks, indicating that multiple workflows were triggered.
**Request**
```http
GET /api/v1/items/98a480ad8d7444a8a10ef547cd8594eb/tasks HTTP/1.1
Accept: application/json
Authorization: Bearer <access token>
Cache-Control: no-cache
Host: farmmaps.awacc.nl
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
```
**Response**
```http
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Sat, 25 Apr 2020 14:34:01 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: no-store,no-cache
Pragma: no-cache
Content-Encoding: br
Vary: Accept-Encoding
[
{
"taskType":"vnd.farmmaps.task.workflow",
"code":"52173fb5bf1146efa276ecfe1fda5f60",
"state":1,
"started":"2020-04-25T13:55:28.58133",
"finished":"2020-04-25T13:55:28.750055"
},
{
"taskType":"vnd.farmmaps.task.vandersat",
"code":"0586f0f867014d55a90bfe6adf7416d5",
"state":1,
"started":"2020-04-25T14:00:36.221527",
"finished":"2020-04-25T14:02:18.314088"
},
{ "taskType":"vnd.farmmaps.task.watbal",
"code":"f33cfaff01644313b8de73dcfd42bc20",
"state":3,
"started":"2020-04-25T14:10:46.201482"
},
{
"taskType":"vnd.farmmaps.task.workflow",
"code":"2b76caf17f6d4f5eb497f7ba7609a08c",
"state":1,
"started":"2020-04-25T13:55:41.278677",
"finished":"2020-04-25T13:55:41.442544"
},
{
"taskType":"vnd.farmmaps.task.crprec",
"code":"0ddd635c244c47b8a182aa5f8dddb5db",
"state":1,
"started":"2020-04-25T13:55:33.609931",
"finished":"2020-04-25T13:55:34.464812"
},
{
"taskType":"vnd.farmmaps.task.bofek",
"code":"e95910401dd8493d8483ebde56f8814c",
"state":1,
"started":"2020-04-25T13:55:33.135703",
"finished":"2020-04-25T13:55:34.620735"
},
{
"taskType":"vnd.farmmaps.task.tipstar",
"code":"172f42ece9004013aaf2b6a56b716e39",
"state":1,
"started":"2020-04-25T13:55:33.903717",
"finished":"2020-04-25T13:55:34.636575"
},
{
"taskType":"vnd.farmmaps.task.soilcompaction",
"code":"1667fce90d954b1b90d9997e5e642e8d",
"state":1,
"started":"2020-04-25T13:55:33.575171",
"finished":"2020-04-25T13:55:34.767101"
},
{
"taskType":"vnd.farmmaps.task.ahn",
"code":"23b1e66325134e8cbd778d57865590f2",
"state":1,
"started":"2020-04-25T13:55:33.465586",
"finished":"2020-04-25T13:55:35.170389"
},
{
"taskType":"vnd.farmmaps.task.tipstar",
"code":"80bcf30893614ad288fe8409cee5f9b1",
"state":1,
"started":"2020-04-25T13:55:36.893606",
"finished":"2020-04-25T13:55:37.331862"
},
{
"taskType":"vnd.farmmaps.task.workflow",
"code":"c4c7d23b6e534a7e9b5dfb79847ea449",
"state":1,
"started":"2020-04-25T13:55:37.237456",
"finished":"2020-04-25T13:55:37.567148"
},
{
"taskType":"vnd.farmmaps.task.trijntje",
"code":"e29539ed7a014dfcb739ad3d87298cc2",
"state":1,
"started":"2020-04-25T13:55:44.271105",
"finished":"2020-04-25T13:55:45.64174"
},
{
"taskType":"vnd.farmmaps.task.satellite",
"code":"b5a549be975e4196972806157ee7f2cd",
"state":1,
"started":"2020-04-25T13:55:33.778997",
"finished":"2020-04-25T13:57:13.811563"
},
{
"taskType":"vnd.farmmaps.task.shadow",
"code":"a1daf49cf08142c289825def78986027",
"state":1,
"started":"2020-04-25T13:55:33.153348",
"finished":"2020-04-25T13:56:14.590014"
}
]
```
**Troubleshooting**
|Status code|Description|
|---|---|
|401|Error: Not authenticated|
|403|Error: No READ permissions in parent item|
|404|Error: Parent item not found|
### Polling a single task
Polling for a siingle task can be done by appending g the `ItemTaskCode` to the task URL:
**Request**
```http
GET /api/v1/items/98a480ad8d7444a8a10ef547cd8594eb/tasks/a1daf49cf08142c289825def78986027 HTTP/1.1
Accept: application/json
Authorization: Bearer <access token>
User-Agent: PostmanRuntime/7.24.1
Cache-Control: no-cache
Postman-Token: dafc3b2a-35be-4f68-b398-ddab1f36864f
Host: farmmaps.awacc.nl
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
```
**Response**
```http
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Sat, 25 Apr 2020 15:02:05 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: no-store,no-cache
Pragma: no-cache
Content-Encoding: br
Vary: Accept-Encoding
{
"taskType":"vnd.farmmaps.task.shadow",
"code":"a1daf49cf08142c289825def78986027",
"state":1,
"started":"2020-04-25T13:55:33.153348",
"finished":"2020-04-25T13:56:14.590014"
}
```
**Troubleshooting**
|Status code|Description|
|---|---|
|401|Error: Not authenticated|
|403|Error: No READ permissions in parent item|
|404|Error: Parent item not found|
## What's next?
Now that you know how to run a task, you can start running specialized tasks to create or process your own data.
Head over to the [Task Examples](/wiki/Home#task-examples)

5
Upload-a-file.md
View File

@ -1,6 +1,7 @@
# Uploading a file
This page documents how to upload a file through the FarmMaps API.
The FarmMaps file upload API follows the [Google Drive API](https://developers.google.com/drive/api/v3/manage-uploads#resumable) loosely.
The FarmMaps file upload API follows the [Google Drive API](https://developers.google.com/drive/api/v3/manage-uploads#resumable) loosely.
Files smaller than 2 MB are uploaded in a single part, larger files need to be split into parts of 1 MB.
The workflow for uploading a file is as follows:
* Register the file for upload.
@ -13,7 +14,7 @@ The workflow for uploading a file is as follows:
### Registering the upload
Before a file can be uploaded it needs to be registered. Files smaller than 2 MB can be uploaded in one chunk.
Bigger files need to be uploaded in chunks of 1 MB. The uploaded files are structured in a hierarchy, so we'll specify a `parentCode` to identify its parent.
We'll first retrieve this parentcode using a GET request.
We'll first retrieve this parentcode using a GET request to the "my_drive" (root)folder.
**Request**
Replace `<acces token>` with your actual token.