package jiveplot #198

sklandrausis · 2019-04-04T11:23:51Z

JIVE developed packages j2ms2, tConvert and jplotter could be include.
Packages j2ms2 allow JIVE developed correlator SFXC output .cor file to be converted to ms files
tConvert provide ms convention to fits that can be loaded into CASA, AIPS.

packages jplotter allow to create inspection plots for VLBI data correlation using ms file created from SFXC .cor file

gijzelaerr · 2019-04-04T11:25:46Z

Can you supply urls to the projects and installation instructions?

sklandrausis · 2019-04-04T12:38:42Z

Installation instructions

CASACORE
git clone http://github.com/casacore/casacore.git
cd casacore
mkdir build
cd build
cmake -DUSE_FFTW3=ON -DCMAKE_INSTALL_PREFIX=/path/to/eee -DUSE_OPENMP=ON USE_HDDF5=ON -DBUILD_PYTHON=ON -DUSE_THREADS=ON ..
make
make install
Python casacore
git clone https://github.com/casacore/python-casacore
cd python-casacore
sudo pip install --upgrade pip
sudo pip install setuptools
unset LC_CTYPE
sudo python setup.py build_ext -I/path/to/eee/include -L/path/to/eee/lib install
mkdir -p /path/to/eee/share/casacore/data
echo '#!/bin/sh

This is the directory where the ephemerides will

be stored

measdir=/path/to/eee/share/casacore/data

attempt to move to that directory

echo cd "${measdir}"
cd "${measdir}"
if [ $? -ne 0 ]; then
echo "$0: Failed to cd into ${measdir}"
exit 1
fi

remove old data

rm -f WSRT_Measures.ztar

ftp over the new stuff

wget ftp://ftp.astron.nl/outgoing/Measures/WSRT_Measures.ztar 2>&1
if [ $? -ne 0 ]; then
echo "$0: Failed to retrieve latest measures data from WSRT"
exit 1
fi

unzip + untar

tar zvxf WSRT_Measures.ztar
if [ $? -ne 0 ]; then
echo "$0: Failed to unbzip2/extract ephemeris data!"
exit 1
fi
' > PREFIX/eee/bin/update_measure_data.sh
chmod +x /path/to/eee/bin/update_measure_data.sh
crontab -l | { cat; echo "0 0 * * sun /home/root2/postCorrelation/update_measure_data.sh | mail -s 'ephemeris update results' [email protected]"; } | crontab -
3. VEX parser
wget http://www.jive.eu/~verkout/myvex.tar.gz
tar zxvf myvex.tar.gz
cd myvex
make

j2ms2 + tConvert
wget http://www.jive.eu/~verkout/jive-casa-snapshot.tar.gz
tar zxvf jive-casa-snapshot.tar.gz
export ROOT=/path/to/eee/src/code/jive
cd code/jive
ln -s implement jive
vim ${ROOT}/system/locations.mak
-VEXROOT := /pcint2/src/myvex-cvs
+VEXROOT := /path/to/eee/src/myvex
```
 -FFTW3ROOT = /pcint2
 +FFTW3ROOT = /usr

 -CASAROOT = /pcint2/src/casacore-2.0.1-root
 +CASAROOT = /path/to/eee
  
 -FFTWROOT = /pcint2
 +FFTWROOT = /usr
  
 -CFITSIOROOT = /pcint2
 +CFITSIOROOT = /usr
```

cd implement
make -f Makefile B2B=64
cd ../apps/
make -f Makefile B2B=64

Python PGPLOT
git clone https://github.com/haavee/ppgplot
cd ppgplot
sudo python setup.py build
jplotter
git clone https://github.com/haavee/jiveplot.git
Create ~/.pythonrc + add PYTHONSTARTUP to ~/.profile
echo '# startup script for python to enable
saving of interpreter history and

enabling name completion

import needed modules

import atexit
import os
import readline
import rlcompleter

where is history saved

historyPath = os.path.expanduser("~/.pyhistory")

handler for saving history

def save_history(historyPath=historyPath):
import readline
readline.write_history_file(historyPath)

read history, if it exists

if os.path.exists(historyPath):
readline.read_history_file(historyPath)

register saving handler

atexit.register(save_history)

enable completion

readline.parse_and_bind('tab: complete')

cleanup

del os, atexit, readline, rlcompleter, save_history, historyPath' > ~/.pythonrc

Final step: make Python read the startup file at startup

$ echo "export PYTHONSTARTUP=${HOME}/.pythonrc" >> ~/.profile

gijzelaerr · 2019-04-04T12:40:01Z

all files on http://www.jive.eu/~verkout/ give a 403 error.

sklandrausis · 2019-04-04T13:09:07Z

i did no know that Joint Institute for VLBI in Europe web page jive.eu currenty is under construction
maybe after some time theirs page will be fixed, but i can send source to email.

sklandrausis · 2019-04-07T11:51:45Z

I checked today all the links work.

gijzelaerr · 2019-04-08T09:08:07Z

@Athanaseus do you want to have a look?

Athanaseus · 2019-04-08T09:10:02Z

Roger that @gijzelaerr

Athanaseus · 2019-04-10T12:18:53Z

Hi @sklandrausis, this also implies that a VEXparser packgage should be independently build and added to KERN?

sklandrausis · 2019-04-10T12:32:18Z

Yes

Athanaseus · 2019-04-23T13:43:58Z

@sklandrausis

j2ms2 + tConvert

After I untar, the resulting directory code has only two directories, jive & vlbi.

According to the instructions, it seems to me only the jive is being used, let me know if I'm missing something.

My approach was to perhaps split it into two say: j2ms2-jive and j2ms2-vlbi.

sklandrausis · 2019-04-23T16:02:49Z

Yes to install only jive directory is used
I am no sure, but probotly probable some files from VLBI is include to files in jive directory

gijzelaerr · 2020-05-14T07:30:50Z

I had a little chat with Harro Verkouter, which explained j2ms2 and tConvert are not publicly available yet, and those snapshots are outdated.

Only https://github.com/haavee/jiveplot is available, so let's see if we can quickly package that one. If j2ms2 and tConvert come available in the future we can maybe package them around that time.

haavee · 2020-05-14T10:45:37Z

Please contact me (Harro Verkouter/JIVE) about installing/packaging questions in the future.

We are the owners and original authors and would be happy to update you in great detail about dependencies and installation instructions and answer questions, if and when the software will be publicly available.

Athanaseus · 2020-05-14T12:19:20Z

Thank you so much @haavee.

Is there any public release that I can start working on?

haavee · 2020-05-14T12:31:33Z

Unfortunately no. The code was reorganized, put under git (local repo) control and the build system was rewritten to use CMake so any attempt at packaging the above linked-to version would be a wast of anyone's time. (Especially since I've already made a recipe for building a container image from the source code ...)
Eventually the new code will be made publicly available, either through my personal github or via https://github.com/jive-vlbi (my money is on the latter).

Athanaseus · 2020-05-14T12:44:15Z

Thanks @haavee, I'll await the public announcement of that release then.

gijzelaerr · 2020-05-14T13:06:51Z

Hi @haavee , I think @Athanaseus is refering to jiveplot. It is available and seems to work, but there is no release made on github. We prefer to package software that has a version, so there are no ambiguities about bug tracking and issue resolving.

haavee · 2020-05-14T13:12:47Z

Oh sorry! I did not realize that, my bad! Up until now there have not been any releases; I can make one because it is the proper way to go.

Tada: https://github.com/haavee/jiveplot/releases

gijzelaerr · 2021-05-17T06:22:32Z

@Athanaseus did you make progress adding jiveplot?

Athanaseus · 2021-05-17T14:16:25Z

I'm working on this right now. It seems like the code is not fully ported to python3 yet?
https://github.com/haavee/jiveplot/search?q=print

gijzelaerr · 2021-05-17T14:18:06Z

could be, @haavee what is the py3 status of jiveplot?

haavee · 2021-05-17T14:39:23Z

I have a Py3 branch that I've tested internally and seems to be working. However, with Python you're never quite sure you've tested all code paths, but smoke tests were passed.
I don't think I've pushed that branch but certainly could give that a go.

Actually - the branch is on the public github already:
https://github.com/haavee/jiveplot/tree/python23compat

haavee · 2021-05-17T14:41:16Z

I checked today all the links work.

We've had some issues with the webserver configuration lately, sorry for the inconvenience. The pages themselves were never at risk.

Athanaseus · 2021-05-18T08:55:58Z

Thanks @haavee, I'm testing the Py3 patch. Can I ask:
Why jiveplot has no setup.py file?
Is ppgplot a dependency or can we still use giza?

$ ./jplotter 
Traceback (most recent call last):
  File "./jplotter", line 51, in <module>
    import command, jplotter
  File "jplotter.py", line 398, in <module>
    import copy, re, math, operator, itertools, plotiterator, ppgplot, datetime, os, subprocess, numpy, parsers, imp, time
ModuleNotFoundError: No module named 'ppgplot

Solved by:
pip install git+https://github.com/haavee/ppgplot.git

$ ./jplotter 
parsers.py:2655: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if tok(s).type is not 'comma':
+++++++++++++++++++++ Welcome to cli +++++++++++++++++++
$Id: command.py,v 1.16 2015-11-04 13:30:10 jive_cc Exp $
  'exit' exits, 'list' lists, 'help' helps 
jcli>

haavee · 2021-05-18T10:09:42Z

Hi @Athanaseus,
jiveplot does not have a setup.py because I never intended it as a package, but as a standalone command line program, e.g. installed in /usr/local/bin. The structure of the project therefore never followed the proper model.

If the dependency issue(s) can be fixed with a setup.py, that would be an option, although I'm not sure how easy it would be to backport the proper project structure into the project.

As to that syntax warning: I have not seen that one but that is probably due to not having access to a modern Python environment (the latest Py3 I have access to is 3.6.5)

Athanaseus · 2021-05-18T13:45:19Z

@haavee I think packaging can still be possible even if it's a command-line program.
It will be easy to restructure from looking at the source code.
Also, this will surely resolve the dependency issues.
It's not that complicated, so if you don't mind I can set it up for you.

haavee · 2021-05-18T13:47:33Z

Oh that would be generous if you could help with that!

Athanaseus · 2021-05-18T13:52:33Z

Cool. I using Py23 branch to set it up. Can you please add me to the project and I will push the branch for you to review.

haavee · 2021-05-18T15:35:24Z

Can't you fork-and-make-a-PR?

Athanaseus · 2021-05-19T06:31:37Z

Good idea. Here check it out haavee/jiveplot#17

gijzelaerr assigned gijzelaerr and Athanaseus and unassigned gijzelaerr Apr 8, 2019

gijzelaerr changed the title ~~No packages j2ms2, tConvert and jplotter~~ package jiveplot May 14, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

package jiveplot #198

package jiveplot #198

sklandrausis commented Apr 4, 2019

gijzelaerr commented Apr 4, 2019

sklandrausis commented Apr 4, 2019

gijzelaerr commented Apr 4, 2019

sklandrausis commented Apr 4, 2019

sklandrausis commented Apr 7, 2019

gijzelaerr commented Apr 8, 2019

Athanaseus commented Apr 8, 2019

Athanaseus commented Apr 10, 2019

sklandrausis commented Apr 10, 2019

Athanaseus commented Apr 23, 2019

sklandrausis commented Apr 23, 2019

gijzelaerr commented May 14, 2020

haavee commented May 14, 2020

Athanaseus commented May 14, 2020

haavee commented May 14, 2020

Athanaseus commented May 14, 2020

gijzelaerr commented May 14, 2020

haavee commented May 14, 2020 •

edited

Loading

gijzelaerr commented May 17, 2021

Athanaseus commented May 17, 2021

gijzelaerr commented May 17, 2021

haavee commented May 17, 2021 •

edited

Loading

haavee commented May 17, 2021

Athanaseus commented May 18, 2021 •

edited

Loading

haavee commented May 18, 2021

Athanaseus commented May 18, 2021

haavee commented May 18, 2021

Athanaseus commented May 18, 2021

haavee commented May 18, 2021

Athanaseus commented May 19, 2021

package jiveplot #198

package jiveplot #198

Comments

sklandrausis commented Apr 4, 2019

gijzelaerr commented Apr 4, 2019

sklandrausis commented Apr 4, 2019

This is the directory where the ephemerides will

be stored

attempt to move to that directory

remove old data

ftp over the new stuff

unzip + untar

enabling name completion

import needed modules

where is history saved

handler for saving history

read history, if it exists

register saving handler

enable completion

cleanup

Final step: make Python read the startup file at startup

gijzelaerr commented Apr 4, 2019

sklandrausis commented Apr 4, 2019

sklandrausis commented Apr 7, 2019

gijzelaerr commented Apr 8, 2019

Athanaseus commented Apr 8, 2019

Athanaseus commented Apr 10, 2019

sklandrausis commented Apr 10, 2019

Athanaseus commented Apr 23, 2019

sklandrausis commented Apr 23, 2019

gijzelaerr commented May 14, 2020

haavee commented May 14, 2020

Athanaseus commented May 14, 2020

haavee commented May 14, 2020

Athanaseus commented May 14, 2020

gijzelaerr commented May 14, 2020

haavee commented May 14, 2020 • edited Loading

gijzelaerr commented May 17, 2021

Athanaseus commented May 17, 2021

gijzelaerr commented May 17, 2021

haavee commented May 17, 2021 • edited Loading

haavee commented May 17, 2021

Athanaseus commented May 18, 2021 • edited Loading

haavee commented May 18, 2021

Athanaseus commented May 18, 2021

haavee commented May 18, 2021

Athanaseus commented May 18, 2021

haavee commented May 18, 2021

Athanaseus commented May 19, 2021

haavee commented May 14, 2020 •

edited

Loading

haavee commented May 17, 2021 •

edited

Loading

Athanaseus commented May 18, 2021 •

edited

Loading