"This document shows how to generate a report for a project."
"This notebook shows how to generate a report for a project.\n",
"\n",
"The data is retrieved through the *TMSS APIs* and it is analysed and visualised using the *Pandas* library.\n",
"\n",
"---"
]
},
{
...
...
@@ -17,13 +21,23 @@
"source": [
"### Prerequirements\n",
"\n",
"The following modules are needed before proceeding:"
"Before proceeding you need to import some modules, as well as specify some configurations."
]
},
{
"cell_type": "markdown",
"id": "e36852c6",
"metadata": {},
"source": [
"#### Imports\n",
"\n",
"The Pandas and Requests libraries are required."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "48b766e7",
"id": "ae6bff14",
"metadata": {},
"outputs": [],
"source": [
...
...
@@ -31,6 +45,35 @@
"import requests"
]
},
{
"cell_type": "markdown",
"id": "6ada730f",
"metadata": {},
"source": [
"#### Configs\n",
"\n",
"Your authentication credentials are needed to perform HTTP requests to the TMSS APIs."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "48b766e7",
"metadata": {},
"outputs": [],
"source": [
"BASE_URL = 'http://localhost:8000/api' # TMSS API endpoint\n",
"auth = ('test', 'test') # username and password"
]
},
{
"cell_type": "markdown",
"id": "340c050b",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"id": "9812780a",
...
...
@@ -40,7 +83,7 @@
"\n",
"To retrieve the data, you need to perform a GET request to the following endpoint: `http://127.0.0.1:8000/api/project/<project>/report`\n",
"\n",
"This can be done by using the `requests` module. You need to provide your authentication credentials in the `auth` parameter. Since the response will be a JSON object, you can simply store the result of `response.json()` as follows:"
"This can be done by using the `requests` module. To perform the request, you need to provide your target project, by specifying its *id* in the `project` variable, and to pass your authentication credentials in the `auth` parameter. Since the response will be a JSON object, you can simply store the result of `response.json()` in the `result` variable."
"Once you have retrieved the data, you need to apply some preprocessing. In the following snippet, we extract the data into variables that will be used afterwards."
"Once you have retrieved the data, you need to extract it in a proper way. In the following snippet, we do such operation by defining some variables that will be used afterwards."
]
},
{
"cell_type": "code",
"execution_count": 135,
"execution_count": 4,
"id": "d1b58c3a",
"metadata": {},
"outputs": [],
...
...
@@ -188,7 +230,17 @@
"id": "883d53a9",
"metadata": {},
"source": [
"You can now use Pandas for the data analysis and visualisation parts."
"You can now use a library (i.e., Pandas) for the data analysis and visualisation parts.\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "d2e62c29",
"metadata": {},
"source": [
"## Create tables\n"
]
},
{
...
...
@@ -196,14 +248,14 @@
"id": "c9b7d51e",
"metadata": {},
"source": [
"## Create a report\n",
"### Summary Table\n",
"\n",
"You can create a unique table within all the data related to a project. It might be convenient to create a different `DataFrame` for each variable of the previous step, as they could be used for subsequent analysis later."
"To better visualise the data, you could plot it in several ways. For example, you might want to observe the differences between all of the four durations retrieved, as follows:"
"To summarise both finished and failed SchedulingUnitBlueprints, you can concatenate the prior DataFrames as well as adding a new column to distinguish them in the new DataFrame:"
]
},
{
"cell_type": "code",
"execution_count": 254,
"id": "a2e8f9cb",
"execution_count": 14,
"id": "d709f04e",
"metadata": {},
"outputs": [
{
...
...
@@ -746,7 +864,7 @@
"36 another_horrible_sub 200.000003 failed"
]
},
"execution_count": 254,
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
...
...
@@ -762,7 +880,7 @@
},
{
"cell_type": "markdown",
"id": "3be9b79e",
"id": "b7e56fc8",
"metadata": {},
"source": [
"Then, you can plot a bar graph discriminting colors by status:"
This document shows how to generate a report for a project.
This notebook shows how to generate a report for a project.
The data is retrieved through the *TMSS APIs* and it is analysed and visualised using the *Pandas* library.
---
%% Cell type:markdown id:ec22618d tags:
### Prerequirements
The following modules are needed before proceeding:
Before proceeding you need to import some modules, as well as specify some configurations.
%% Cell type:code id:48b766e7 tags:
%% Cell type:markdown id:e36852c6 tags:
#### Imports
The Pandas and Requests libraries are required.
%% Cell type:code id:ae6bff14 tags:
``` python
importpandasaspd
importrequests
```
%% Cell type:markdown id:6ada730f tags:
#### Configs
Your authentication credentials are needed to perform HTTP requests to the TMSS APIs.
%% Cell type:code id:48b766e7 tags:
``` python
BASE_URL='http://localhost:8000/api'# TMSS API endpoint
auth=('test','test')# username and password
```
%% Cell type:markdown id:340c050b tags:
---
%% Cell type:markdown id:9812780a tags:
## Retrieve the data
To retrieve the data, you need to perform a GET request to the following endpoint: `http://127.0.0.1:8000/api/project/<project>/report`
This can be done by using the `requests` module. You need to provide your authentication credentials in the `auth` parameter. Since the response will be a JSON object, you can simply store the result of `response.json()`as follows:
This can be done by using the `requests` module. To perform the request, you need to provide your target project, by specifying its *id* in the `project` variable, and to pass your authentication credentials in the `auth` parameter. Since the response will be a JSON object, you can simply store the result of `response.json()`in the `result` variable.
Once you have retrieved the data, you need to apply some preprocessing. In the following snippet, we extract the data into variables that will be used afterwards.
Once you have retrieved the data, you need to extract it in a proper way. In the following snippet, we do such operation by defining some variables that will be used afterwards.
%% Cell type:code id:d1b58c3a tags:
``` python
project_id=result['project']# Project id
quota=result['quota']# Allocated resources
durations=dict(list(result['durations'].items())[:4])# Durations are the first 4 elements in the JSON object.
You can now use Pandas for the data analysis and visualisation parts.
You can now use a library (i.e., Pandas) for the data analysis and visualisation parts.
---
%% Cell type:markdown id:d2e62c29 tags:
## Create tables
%% Cell type:markdown id:c9b7d51e tags:
## Create a report
### Summary Table
You can create a unique table within all the data related to a project. It might be convenient to create a different `DataFrame` for each variable of the previous step, as they could be used for subsequent analysis later.
%% Cell type:code id:8a0a7ed9 tags:
``` python
# Create a DataFrame for each data you want to summarise
<pandas.io.formats.style.Styler at 0x7f4c158faa90>
<pandas.io.formats.style.Styler at 0x7f6bb40e6f28>
%% Cell type:markdown id:ddfa7824 tags:
%% Cell type:markdown id:8f9b1da2 tags:
Note that for the other values, you can follow a similar procedure as illustrated by the following sections.
For the other values, you can follow a similar procedure:
%% Cell type:markdown id:c2f231ba tags:
%% Cell type:code id:aec5ebb1 tags:
### Quota table
%% Cell type:code id:1b94ce76 tags:
``` python
# Create a DataFrame for quota
df_quota=pd.DataFrame(quota)
df_quota.set_index('id')
```
%% Output
resource_type_id value
id
4 LTA Storage 1000.0
11 LTA Storage 2400.0
%% Cell type:code id:a2621ab9 tags:
%% Cell type:markdown id:47f4503c tags:
``` python
df_saps=pd.DataFrame(saps)
df_saps.set_index('sap_name')
```
### SchedulingUnitBlueprints
%% Output
%% Cell type:markdown id:93b336e7 tags:
total_exposure
sap_name
sap_1 340.0
sap_2 195.0
sap_3 235.0
#### Finished SUBs
%% Cell type:code id:192cea33 tags:
%% Cell type:code id:10537659 tags:
``` python
# Create a DataFrame for finished SUBs
df_subs_finished=pd.DataFrame(subs_finished)
df_subs_finished.set_index('id')
```
%% Output
name duration
id
8 amazing_sub 600.000003
21 another_amazing_sub 800.000003
%% Cell type:code id:02d2a788 tags:
%% Cell type:markdown id:c67ef360 tags:
#### Failed SUBs
%% Cell type:code id:6487f39e tags:
``` python
# Create a DataFrame for failed SUBs
df_subs_failed=pd.DataFrame(subs_failed)
df_subs_failed.set_index('id')
```
%% Output
name duration
id
12 horrible_sub 600.000003
36 another_horrible_sub 200.000003
%% Cell type:markdown id:265697ce tags:
### SAPs
%% Cell type:code id:669389c5 tags:
``` python
# Create a DataFrame for SAPs
df_saps=pd.DataFrame(saps)
df_saps.set_index('sap_name')
```
%% Output
total_exposure
sap_name
sap_1 340.0
sap_2 195.0
sap_3 235.0
%% Cell type:markdown id:193c3998 tags:
---
%% Cell type:markdown id:b7a4b0b9 tags:
### Create a plot
## Create a plot
To better visualise the data, you could plot it in several ways. For example, you might want to observe the differences between all of the four durations retrieved, as follows:
To summarise both finished and failed SchedulingUnitBlueprints, you can concatenate the prior DataFrames as well as adding a new column to distinguish them in the new DataFrame:
%% Cell type:code id:a2e8f9cb tags:
%% Cell type:code id:d709f04e tags:
``` python
# Add a status column to differentiate colors later
df_subs_finished['status']='finished'
df_subs_failed['status']='failed'
# Create a new DataFrame, within index sorting, as a concatenation of finished and failed SUBs.
