Skip to content
Snippets Groups Projects
Commit 4641c9fb authored by Stefano Di Frischia's avatar Stefano Di Frischia
Browse files

L2SS-235: add readme file and minor bug fixing

parent 55d525f3
No related branches found
No related tags found
1 merge request!108Resolve L2SS-235 "Archive pcc attribute"
Show whitespace changes
Inline Side-by-side
devices/toolkit/README.md 0 → 100644
+ 42
− 0
View file @ 4641c9fb
# Tango Archiving Framework
The Archiver class in archiver.py defines the methods to manage the device attributes archiving allowed by Tango.
The main components (and the relative Docker containers) are:
- Configuration Manager (container: hdbpp-cm): Device server that assists in adding, modifying, moving, deleting an Attribute to/from the archiving system
- Event Subscriber (container: hdbpp-es): The EventSubscriber TANGO device server, is the archiving system engine. On typical usage, it will subscribe to archive events on request by the ConfigurationManager device. The EventSubscriber is designed to start archiving all the already configured Attributes, even if the ConfigurationManager is not running. Moreover, being a TANGO device, the EventSubscriber configuration can be managed with Jive.
- Archiving DBMS (container: archiver-maria-db): Specific Database devoted to storing attribute values.
- (Optional) HDB++ Viewer (container: hdbpp-viewer): Standalone JAVA application designed to monitor signals coming from database
## Archiver creation
When an Archiver object is created, we can define three of its properties:
- the ConfigurationManager name (Tango namespace)
- at least one EventSubscriber name (Tango namespace)
- the default context archiving for the subscribers. This means that a default archiving strategy will be applied to
all the attributes. Of course this strategy can be tuned individually for each attribute if needed.
Archiving strategies are ['ALWAYS','RUN','SHUTDOWN','SERVICE']
- ALWAYS:always stored
- RUN:stored during run
- SHUTDOWN:stored during shutdown
- SERVICE:stored during maintenance activities
## Add an attribute
When adding an attribute to the archiving framework, we must define the following properties:
- the EventSubscriber name that will take charge of the attribute
- the archiving strategy (4 options defined above)
- the attribute polling period (it should have been already defined in TangoDB)
- the archive event period (MOST IMPORTANT, it defines the frequency rate at which an attribute is archived in the DBMS)
It is important to understand that, when an attribute is successfully added to the EventSubscriber list, the archiving begins without an explicit 'Start' command, rather it follows the archiving strategy already defined.
The 'Start' command is used instead during a session when an attribute has been paused/stopped for any reason, or it has raised some kind of issue.
## Difference between Stop and Remove an attribute
When stopping an attribute archiving, the framework does not remove it from the list.
This means that archiving is stopped for the current session, but if the device is restarted, the attribute archiving will be restarted as well.
In order to definitely stop the archiving, the attribute must be removed from the attribute list.
## Update an attribute
If we want to update the archiving properties of an attribute (e.g. the archive event period), there is a relative method.
It must be noted that the updating is not istantaneous because, following the framework architecture, an attribute must be first removed from the EventSubscriber list and then re-added with the new properties.
devices/toolkit/archiver.py
+ 1
− 1
View file @ 4641c9fb
...@@ -129,7 +129,7 @@ class Archiver(): ...@@ -129,7 +129,7 @@ class Archiver():
else: else:
print('Attribute %s not found!' % attribute_name) print('Attribute %s not found!' % attribute_name)
def check_attribute_in_archiving_list(self, attribute_name:str): def check_and_add_attribute_in_archiving_list(self, attribute_name:str):
""" """
Check if an attribute is in the archiving list Check if an attribute is in the archiving list
""" """
......
jupyter-notebooks/PCC_archive_attribute.ipynb
+ 1
− 1
View file @ 4641c9fb
%% Cell type:code id:42e7f25a tags: %% Cell type:code id:42e7f25a tags:
``` python ``` python
import sys, time import sys, time
import numpy as np import numpy as np
sys.path.append('/hosthome/tango/devices') sys.path.append('/hosthome/tango/devices')
from toolkit.archiver import Archiver,Retriever from toolkit.archiver import Archiver,Retriever
from toolkit.archiver_base import * from toolkit.archiver_base import *
from matplotlib import pyplot as plt from matplotlib import pyplot as plt
``` ```
%% Cell type:code id:1f025912 tags: %% Cell type:code id:1f025912 tags:
``` python ``` python
from common.lofar_environment import isProduction from common.lofar_environment import isProduction
print(isProduction()) print(isProduction())
``` ```
%% Cell type:code id:e0656e2d tags: %% Cell type:code id:e0656e2d tags:
``` python ``` python
# Define an attribute for archiving # Define an attribute for archiving
device_name = 'LTS/PCC/1' device_name = 'LTS/PCC/1'
d=DeviceProxy(device_name) d=DeviceProxy(device_name)
state = str(d.state()) state = str(d.state())
print(device_name,'is',state) print(device_name,'is',state)
archiver = Archiver() archiver = Archiver()
# Attribute chosen to be archived # Attribute chosen to be archived
attr_name = 'rcu_temperature_r' attr_name = 'rcu_temperature_r'
attr_fq_name = str(device_name+'/'+attr_name).lower() attr_fq_name = str(device_name+'/'+attr_name).lower()
``` ```
%% Cell type:code id:153d9420 tags: %% Cell type:code id:153d9420 tags:
``` python ``` python
# Print the list of the attributes in the event subscriber # Print the list of the attributes in the event subscriber
# If any attribute is present, its archiving will begin when device will reach ON state, # If any attribute is present, its archiving will begin when device will reach ON state,
# Otherwise, attribute will be added to the list at the device initializing phase only in PRODUCTION mode # Otherwise, attribute will be added to the list at the device initializing phase only in PRODUCTION mode
archiver.get_subscriber_attributes() archiver.get_subscriber_attributes()
``` ```
%% Cell type:code id:2ebb00f8 tags: %% Cell type:code id:2ebb00f8 tags:
``` python ``` python
# Start the device # Start the device
if state == "OFF": if state == "OFF":
if isProduction(): if isProduction():
archiver.check_attribute_in_archiving_list(attr_fq_name) archiver.check_and_add_attribute_in_archiving_list(attr_fq_name)
else: else:
archiver.remove_attribute_from_archiver(attr_fq_name) archiver.remove_attribute_from_archiver(attr_fq_name)
time.sleep(1) time.sleep(1)
d.initialise() d.initialise()
time.sleep(1) time.sleep(1)
state = str(d.state()) state = str(d.state())
if state == "STANDBY": if state == "STANDBY":
d.on() d.on()
state = str(d.state()) state = str(d.state())
if state == "ON": if state == "ON":
print("Device is now in ON state") print("Device is now in ON state")
``` ```
%% Cell type:code id:75163627 tags: %% Cell type:code id:75163627 tags:
``` python ``` python
# Modify attribute archiving features # Modify attribute archiving features
archiver.update_archiving_attribute(attr_fq_name,polling_period=1000,event_period=5000,strategy='RUN') archiver.update_archiving_attribute(attr_fq_name,polling_period=1000,event_period=5000,strategy='RUN')
``` ```
%% Cell type:code id:7814715e tags: %% Cell type:code id:7814715e tags:
``` python ``` python
# Add attribute to the archiving list (starts the archiving if device is running) # Add attribute to the archiving list (starts the archiving if device is running)
# Archiving strategies are ['ALWAYS','RUN','SHUTDOWN','SERVICE'] # Archiving strategies are ['ALWAYS','RUN','SHUTDOWN','SERVICE']
#Read [0] ALWAYS:always stored #Read [0] ALWAYS:always stored
#Read [1] RUN:stored during run #Read [1] RUN:stored during run
#Read [2] SHUTDOWN:stored during shutdown #Read [2] SHUTDOWN:stored during shutdown
#Read [3] SERVICE:stored during maintenance activities #Read [3] SERVICE:stored during maintenance activities
archiver.add_attribute_to_archiver(attr_fq_name, polling_period=1000, event_period=1000, strategy='RUN') archiver.add_attribute_to_archiver(attr_fq_name, polling_period=1000, event_period=1000, strategy='RUN')
``` ```
%% Cell type:code id:52a27abb tags: %% Cell type:code id:52a27abb tags:
``` python ``` python
# Stop the attribute archiving but do not remove it from the list # Stop the attribute archiving but do not remove it from the list
# This means that archiving is stopped for the current session, but if the device is restarted, # This means that archiving is stopped for the current session, but if the device is restarted,
# the attribute archiving will be restarted as well # the attribute archiving will be restarted as well
# In order to definitely stop the archiving, the attribute must be removed from the attribute list (go to last cell) # In order to definitely stop the archiving, the attribute must be removed from the attribute list (go to last cell)
archiver.stop_archiving_attribute(attr_fq_name) archiver.stop_archiving_attribute(attr_fq_name)
``` ```
%% Cell type:code id:c064e337 tags: %% Cell type:code id:c064e337 tags:
``` python ``` python
# Starts the attribute archiving if it was stopped # Starts the attribute archiving if it was stopped
archiver.start_archiving_attribute(attr_fq_name) archiver.start_archiving_attribute(attr_fq_name)
``` ```
%% Cell type:code id:d199916c tags: %% Cell type:code id:d199916c tags:
``` python ``` python
# Initialise the retriever object and print the archived attributes in the database # Initialise the retriever object and print the archived attributes in the database
retriever = Retriever() retriever = Retriever()
retriever.get_all_archived_attributes() retriever.get_all_archived_attributes()
``` ```
%% Cell type:code id:80e2a560 tags: %% Cell type:code id:80e2a560 tags:
``` python ``` python
# Retrieve records in the last n hours (works even with decimals) # Retrieve records in the last n hours (works even with decimals)
# Use alternatively one of the following two methods to retrieve data (last n hours or interval) # Use alternatively one of the following two methods to retrieve data (last n hours or interval)
records= retriever.get_attribute_value_by_hours(attr_fq_name,hours=0.1) records= retriever.get_attribute_value_by_hours(attr_fq_name,hours=0.1)
#records = retriever.get_attribute_value_by_interval(attr_fq_name,'2021-09-01 16:00:00', '2021-09-01 16:03:00') #records = retriever.get_attribute_value_by_interval(attr_fq_name,'2021-09-01 16:00:00', '2021-09-01 16:03:00')
if not records: if not records:
print('Empty result!') print('Empty result!')
else: else:
# Convert DB Array records into Python lists # Convert DB Array records into Python lists
data = build_array_from_record(records,records[0].dim_x_r) data = build_array_from_record(records,records[0].dim_x_r)
# Extract only the value from the array # Extract only the value from the array
array_values = get_values_from_record(data) array_values = get_values_from_record(data)
#records #records
#data #data
#array_values #array_values
``` ```
%% Cell type:code id:64c8e060 tags: %% Cell type:code id:64c8e060 tags:
``` python ``` python
# Extract and process timestamps for plotting purposes # Extract and process timestamps for plotting purposes
def get_timestamps(data,strformat): def get_timestamps(data,strformat):
timestamps = [] timestamps = []
for i in range(len(data)): for i in range(len(data)):
timestamps.append(data[i][0].recv_time.strftime(strformat)) timestamps.append(data[i][0].recv_time.strftime(strformat))
return timestamps return timestamps
timestamps = get_timestamps(data,"%Y-%m-%d %X") timestamps = get_timestamps(data,"%Y-%m-%d %X")
``` ```
%% Cell type:code id:59a0c05c tags: %% Cell type:code id:59a0c05c tags:
``` python ``` python
# Plot of array values # Plot of array values
heatmap = np.array(array_values,dtype=np.float) heatmap = np.array(array_values,dtype=np.float)
fig = plt.figure() fig = plt.figure()
plt.rcParams['figure.figsize'] = [128, 64] plt.rcParams['figure.figsize'] = [128, 64]
#plt.rcParams['figure.dpi'] = 128 #plt.rcParams['figure.dpi'] = 128
ax = fig.add_subplot(111) ax = fig.add_subplot(111)
im = ax.imshow(heatmap, interpolation='nearest',cmap='coolwarm') im = ax.imshow(heatmap, interpolation='nearest',cmap='coolwarm')
ax.set_xlabel('Array index') ax.set_xlabel('Array index')
ax.set_ylabel('Timestamp') ax.set_ylabel('Timestamp')
ax.set_xlim([0,(records[0].dim_x_r)-1]) ax.set_xlim([0,(records[0].dim_x_r)-1])
ax.set_xticks(np.arange(0,records[0].dim_x_r)) ax.set_xticks(np.arange(0,records[0].dim_x_r))
ax.set_yticks(range(0,len(timestamps))) ax.set_yticks(range(0,len(timestamps)))
ax.set_yticklabels(timestamps,fontsize=4) ax.set_yticklabels(timestamps,fontsize=4)
# Comment the previous two lines and uncomment the following line if there are too many timestamp labels # Comment the previous two lines and uncomment the following line if there are too many timestamp labels
#ax.set_yticks(range(0,len(timestamps),10)) #ax.set_yticks(range(0,len(timestamps),10))
ax.set_title('Archived data for '+ attr_fq_name) ax.set_title('Archived data for '+ attr_fq_name)
ax.grid() ax.grid()
cbar = fig.colorbar(ax=ax, mappable=im, orientation='horizontal') cbar = fig.colorbar(ax=ax, mappable=im, orientation='horizontal')
plt.show() plt.show()
``` ```
%% Cell type:code id:a0e8dcab tags: %% Cell type:code id:a0e8dcab tags:
``` python ``` python
# Turn off the device # Turn off the device
d.off() d.off()
# Remove attribute from archiving list # Remove attribute from archiving list
#archiver.remove_attribute_from_archiver(attr_fq_name) #archiver.remove_attribute_from_archiver(attr_fq_name)
#archiver.remove_attributes_by_device(device_name) #archiver.remove_attributes_by_device(device_name)
``` ```
%% Cell type:code id:32ab34a9 tags: %% Cell type:code id:32ab34a9 tags:
``` python ``` python
``` ```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment