Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
T
tango
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Iterations
Wiki
Requirements
Jira issues
Open Jira
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Test cases
Artifacts
Deploy
Releases
Package registry
Container registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Insights
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
LOFAR2.0
tango
Commits
9de9401a
Commit
9de9401a
authored
4 years ago
by
Jan David Mol
Browse files
Options
Downloads
Patches
Plain Diff
Reroute Jupyter Notebook logs to ELK
parent
a31b4e7f
Branches
Branches containing commit
Tags
Tags containing commit
1 merge request
!7
Resolve #2021 "03 16 branched from master elk stack"
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
docker-compose/jupyter.yml
+1
-1
1 addition, 1 deletion
docker-compose/jupyter.yml
docker-compose/jupyter/Dockerfile
+7
-3
7 additions, 3 deletions
docker-compose/jupyter/Dockerfile
docker-compose/jupyter/jupyter-notebook
+28
-0
28 additions, 0 deletions
docker-compose/jupyter/jupyter-notebook
with
36 additions
and
4 deletions
docker-compose/jupyter.yml
+
1
−
1
View file @
9de9401a
...
@@ -33,5 +33,5 @@ services:
...
@@ -33,5 +33,5 @@ services:
-
--timeout=30
-
--timeout=30
-
--strict
-
--strict
-
--
-
--
-
/usr/bin/tini -- jupyter
notebook --port=8888 --no-browser --ip=0.0.0.0 --allow-root --NotebookApp.token= --NotebookApp.password=
-
/usr/bin/tini --
/usr/local/bin/
jupyter
-
notebook --port=8888 --no-browser --ip=0.0.0.0 --allow-root --NotebookApp.token= --NotebookApp.password=
restart
:
on-failure
restart
:
on-failure
This diff is collapsed.
Click to expand it.
docker-compose/jupyter/Dockerfile
+
7
−
3
View file @
9de9401a
ARG
VERSION=latest
ARG
VERSION=latest
FROM
nexus.engageska-portugal.pt/ska-docker/tango-itango:${VERSION}
FROM
nexus.engageska-portugal.pt/ska-docker/tango-itango:${VERSION}
RUN
pip3
install
jupyter
RUN
sudo
pip3
install
jupyter
RUN
pip3
install
ipykernel
RUN
sudo
pip3
install
ipykernel
RUN
pip3
install
jupyter_bokeh
RUN
sudo
pip3
install
jupyter_bokeh
# Configure jupyter_bokeh
# Configure jupyter_bokeh
RUN
sudo mkdir
-p
/usr/share/jupyter /usr/etc
RUN
sudo mkdir
-p
/usr/share/jupyter /usr/etc
...
@@ -16,6 +16,10 @@ COPY ipython-profiles /opt/ipython-profiles/
...
@@ -16,6 +16,10 @@ COPY ipython-profiles /opt/ipython-profiles/
RUN
sudo chown
tango.tango
-R
/opt/ipython-profiles
RUN
sudo chown
tango.tango
-R
/opt/ipython-profiles
COPY
jupyter-kernels /usr/local/share/jupyter/kernels/
COPY
jupyter-kernels /usr/local/share/jupyter/kernels/
# Install patched jupyter executable
RUN
sudo
pip3
install
python-logstash-async
COPY
jupyter-notebook /usr/local/bin/jupyter-notebook
# Add Tini. Tini operates as a process subreaper for jupyter. This prevents kernel crashes.
# Add Tini. Tini operates as a process subreaper for jupyter. This prevents kernel crashes.
ENV
TINI_VERSION v0.6.0
ENV
TINI_VERSION v0.6.0
ADD
https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /usr/bin/tini
ADD
https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /usr/bin/tini
...
...
This diff is collapsed.
Click to expand it.
docker-compose/jupyter/jupyter-notebook
0 → 100755
+
28
−
0
View file @
9de9401a
#!/usr/bin/python3
# -*- coding: utf-8 -*-
# An adjustment of the `jupyter-notebook' executable patched to:
# - log to the ELK stack
#
# We go straight for the notebook executable here, as the "jupyter" command
# execvp's into the requested notebook subcommand, erasing all configuration
# we set here.
import
re
import
sys
from
notebook.notebookapp
import
main
from
logstash_async.handler
import
AsynchronousLogstashHandler
,
LogstashFormatter
import
logging
if
__name__
==
'
__main__
'
:
# log to the tcp_input of logstash in our ELK stack
handler
=
AsynchronousLogstashHandler
(
"
elk
"
,
5959
,
database_path
=
'
/tmp/pending_log_messages.db
'
)
# add to logger of Jupyter traitlets Application. As that logger is configured not to propagate
# messages upward, we need to configure it directly.
logger
=
logging
.
getLogger
(
"
NotebookApp
"
)
logger
.
addHandler
(
handler
)
logger
.
setLevel
(
logging
.
DEBUG
)
sys
.
argv
[
0
]
=
re
.
sub
(
r
'
(-script\.pyw|\.exe)?$
'
,
''
,
sys
.
argv
[
0
])
sys
.
exit
(
main
())
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment