Skip to content
Snippets Groups Projects
Commit 185114a0 authored by Matthijs van der Wild's avatar Matthijs van der Wild
Browse files

move python logic get_memory_fraction to separate library

parent 45ce1906
No related branches found
No related tags found
1 merge request!89move python logic get_memory_fraction to separate library
...@@ -7,6 +7,7 @@ dependencies = [ ...@@ -7,6 +7,7 @@ dependencies = [
"losoto", "losoto",
"numpy", "numpy",
"pandas", "pandas",
"psutil",
] ]
[project.optional-dependencies] [project.optional-dependencies]
......
#!/usr/bin/env python3
import argparse
import json
import psutil
import sys
def main(memory_percentage):
"""
Computes memory_percentage percent of the available virtual memory.
Writes the amount of available memory to a dictionary.
"""
# psutil outputs the memory in bytes. This is converted into
# mebibytes (1 MiB = 2^20 B) to match CWL's ResourceRequirement input.
required_memory = int(
psutil.virtual_memory().available / 2**20 * memory_percentage / 100
)
result = {"memory": required_memory}
with open("./memory.json", "w") as fp:
json.dump(result, fp)
return result
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("memory_percentage", type=int)
main(parser.parse_args().memory_percentage)
...@@ -7,15 +7,15 @@ doc: | ...@@ -7,15 +7,15 @@ doc: |
This is used to ensure that flagging jobs don't exceed the This is used to ensure that flagging jobs don't exceed the
node's resources. node's resources.
baseCommand: baseCommand: query_memory.py
- python3
- query_memory.py
stdout: memory.txt stdout: memory.txt
inputs: inputs:
- id: fraction - id: fraction
type: int type: int
inputBinding:
position: 1
doc: | doc: |
The required fraction of the node's The required fraction of the node's
available memory for a flagging job. available memory for a flagging job.
...@@ -37,20 +37,4 @@ hints: ...@@ -37,20 +37,4 @@ hints:
requirements: requirements:
- class: InlineJavascriptRequirement - class: InlineJavascriptRequirement
- class: InitialWorkDirRequirement
listing:
- entryname: query_memory.py
entry: |
import json
import psutil
# psuitil outputs the memory in bytes.
# This is converted into mebibytes (1 MiB = 2^20 B)
# to match CWL's ResourceRequirement input.
required_memory = int(psutil.virtual_memory().available / 2**20
* $(inputs.fraction) / 100)
print("The available memory is ", required_memory)
result = {"memory" : required_memory}
with open('./memory.json', 'w') as fp:
json.dump(result, fp)
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment