Utility Programs¶
This library also provides some command-line utility programs to work with PVL text.
pvl_translate¶
A program for converting PVL text to a specific PVL dialect.
The pvl_translate
program will read a file with PVL text (any
of the kinds of files that pvl.load()
reads) or STDIN and
will convert that PVL text to a particular PVL dialect. It is not
particularly robust, and if it cannot make simple conversions, it
will raise errors.
usage: pvl_translate [-h] -of {PDS3,ODL,ISIS,PVL,JSON} [--version]
[infile] [outfile]
-
infile
¶
file containing PVL text to translate, defaults to STDIN.
-
outfile
¶
file to write translated PVL to, defaults to STDOUT.
-
-h
,
--help
¶
show this help message and exit
-
-of
{PDS3,ODL,ISIS,PVL,JSON}
,
--output_format
{PDS3,ODL,ISIS,PVL,JSON}
¶ Select the format to create the new file as.
-
--version
¶
show program’s version number and exit
In the examples below will all operate on the file with these contents:
PDS_VERSION_ID = PDS3
/* FILE DATA ELEMENTS */
RECORD_TYPE = FIXED_LENGTH
RECORD_BYTES = 824
LABEL_RECORDS = 1
FILE_RECORDS = 601
/* POINTERS TO DATA OBJECTS */
^IMAGE = 2
/* IMAGE DATA ELEMENTS */
OBJECT = IMAGE
LINES = 600
LINE_SAMPLES = 824
SAMPLE_TYPE = MSB_INTEGER
SAMPLE_BITS = 8
MEAN = 51.67785396440129
MEDIAN = 50.00000
MINIMUM = 0
MAXIMUM = 255
STANDARD_DEVIATION = 16.97019
CHECKSUM = 25549531
END_OBJECT = IMAGE
END
Convert to PDS3 (whitespace and comments get removed):
> pvl_translate -of PDS3 tests/data/pds3/simple_image_1.lbl
PDS_VERSION_ID = PDS3
RECORD_TYPE = FIXED_LENGTH
RECORD_BYTES = 824
LABEL_RECORDS = 1
FILE_RECORDS = 601
^IMAGE = 2
OBJECT = IMAGE
LINES = 600
LINE_SAMPLES = 824
SAMPLE_TYPE = MSB_INTEGER
SAMPLE_BITS = 8
MEAN = 51.67785396440129
MEDIAN = 50.0
MINIMUM = 0
MAXIMUM = 255
STANDARD_DEVIATION = 16.97019
CHECKSUM = 25549531
END_OBJECT = IMAGE
END
Convert to PVL:
> pvl_translate -of PVL tests/data/pds3/simple_image_1.lbl
PDS_VERSION_ID = PDS3;
RECORD_TYPE = FIXED_LENGTH;
RECORD_BYTES = 824;
LABEL_RECORDS = 1;
FILE_RECORDS = 601;
^IMAGE = 2;
BEGIN_OBJECT = IMAGE;
LINES = 600;
LINE_SAMPLES = 824;
SAMPLE_TYPE = MSB_INTEGER;
SAMPLE_BITS = 8;
MEAN = 51.67785396440129;
MEDIAN = 50.0;
MINIMUM = 0;
MAXIMUM = 255;
STANDARD_DEVIATION = 16.97019;
CHECKSUM = 25549531;
END_OBJECT = IMAGE;
END;
Convert to JSON:
> pvl_translate -of JSON tests/data/pds3/simple_image_1.lbl
{"PDS_VERSION_ID": "PDS3", "RECORD_TYPE": "FIXED_LENGTH", "RECORD_BYTES": 824, "LABEL_RECORDS": 1, "FILE_RECORDS": 601, "^IMAGE": 2, "IMAGE": {"LINES": 600, "LINE_SAMPLES": 824, "SAMPLE_TYPE": "MSB_INTEGER", "SAMPLE_BITS": 8, "MEAN": 51.67785396440129, "MEDIAN": 50.0, "MINIMUM": 0, "MAXIMUM": 255, "STANDARD_DEVIATION": 16.97019, "CHECKSUM": 25549531}}
pvl_validate¶
A program for testing and validating PVL text.
The pvl_validate
program will read a file with PVL text (any of
the kinds of files that pvl.load()
reads) and will report
on which of the various PVL dialects were able to load that PVL
text, and then also reports on whether the pvl
library can encode
the Python Objects back out to PVL text.
You can imagine some PVL text that could be loaded, but is not able to be written out in a particular strict PVL dialect (like PDS3 labels).
usage: pvl_validate [-h] [-v] [--version] file [file ...]
-
file
¶
file containing PVL text to validate.
-
-h
,
--help
¶
show this help message and exit
-
-v
,
--verbose
¶
Will report the errors that are encountered. A second v will include tracebacks for non-pvl exceptions.
-
--version
¶
show program’s version number and exit
Validate one file:
> pvl_validate tests/data/pds3/simple_image_1.lbl
PDS3 | Loads | Encodes
ODL | Loads | Encodes
PVL | Loads | Encodes
ISIS | Loads | Encodes
Omni | Loads | Encodes
>
You can see here that the simple_image_1.lbl
file can be
loaded and the resulting Python object encoded with each of the
PVL dialects that the pvl
library knows.
A file with broken PVL text:
> pvl_validate tests/data/pds3/broken/broken1.lbl
PDS3 | does NOT load |
ODL | does NOT load |
PVL | does NOT load |
ISIS | Loads | Encodes
Omni | Loads | Encodes
>
Here, the PVL text in broken1.lbl cannot be loaded by the PDS3, ODL, or PVL dialects, to learn why
use -v
:
> pvl_validate -v tests/data/pds3/broken/broken1.lbl
ERROR: PDS3 load error tests/data/pds3/broken/broken1.lbl (LexerError(...), 'Expecting an Aggregation Block, an Assignment Statement, or an End Statement, but found "=" : line 3 column 7 (char 23)')
ERROR: ODL load error tests/data/pds3/broken/broken1.lbl (LexerError(...), 'Expecting an Aggregation Block, an Assignment Statement, or an End Statement, but found "=" : line 3 column 7 (char 23)')
ERROR: PVL load error tests/data/pds3/broken/broken1.lbl (LexerError(...), 'Expecting an Aggregation Block, an Assignment Statement, or an End Statement, but found "=" : line 3 column 7 (char 23)')
PDS3 | does NOT load |
ODL | does NOT load |
PVL | does NOT load |
ISIS | Loads | Encodes
Omni | Loads | Encodes
This tells us that in these cases, there is a parameter with a missing value. However, the OmniParser (the default, and also what the ISIS dialect uses) has more tolerance for broken PVL text, and is able to load it, and then write valid PVL back out.
Here’s a file which has some PVL text which is valid for some dialects, but not others:
> pvl_validate tests/data/pds3/dates.lbl
PDS3 | Loads | does NOT encode
ODL | Loads | Encodes
PVL | Loads | Encodes
ISIS | Loads | Encodes
Omni | Loads | Encodes
>
Here, pvl_validate
indicates that it can load the file with all of the PVL dialects, and
can encode it back for most. What was the problem:
> pvl_validate -v tests/data/pds3/dates.lbl
ERROR: PDS3 encode error tests/data/pds3/dates.lbl PDS labels should only have UTC times, but this time has a timezone: 01:12:22+07:00
PDS3 | Loads | does NOT encode
ODL | Loads | Encodes
PVL | Loads | Encodes
ISIS | Loads | Encodes
Omni | Loads | Encodes
It indicates that it cannot encode the Python object out to the PDS3 format because it contains a date with a different time zone (which aren’t allowed in a PDS3 Label). So this is an example of how the loaders are a little more permissive, but to really test whether some PVL text is conformant, it also should be able to be encoded.
In this case, if the user wants to write out a valid PDS3 label, they will have to do some work to fix the value.
Validating a bunch of files:
> pvl_validate tests/data/pds3/*lbl
---------------------------------------+-----------+-----------+-----------+-----------+----------
File | PDS3 | ODL | PVL | ISIS | Omni
---------------------------------------+-----------+-----------+-----------+-----------+----------
tests/data/pds3/backslashes.lbl | L E | L E | L E | L E | L E
tests/data/pds3/based_integer1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/dates.lbl | L No E | L E | L E | L E | L E
tests/data/pds3/empty.lbl | L E | L E | L E | L E | L E
tests/data/pds3/float1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/float_unit1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/group1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/group2.lbl | L E | L E | L E | L E | L E
tests/data/pds3/group3.lbl | L E | L E | L E | L E | L E
tests/data/pds3/group4.lbl | L E | L E | L E | L E | L E
tests/data/pds3/namespaced_string1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/negative_float1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/negative_int1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/nested_object1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/nested_object2.lbl | L E | L E | L E | L E | L E
tests/data/pds3/scaled_real1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/sequence1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/sequence2.lbl | L E | L E | L E | L E | L E
tests/data/pds3/sequence3.lbl | L E | L E | L E | L E | L E
tests/data/pds3/sequence_units1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/set1.lbl | L No E | L E | L E | L E | L E
tests/data/pds3/set2.lbl | L No E | L E | L E | L E | L E
tests/data/pds3/simple_image_1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/simple_image_2.lbl | L E | L E | L E | L E | L E
tests/data/pds3/string2.lbl | L E | L E | L E | L E | L E
tests/data/pds3/string3.lbl | L E | L E | L E | L E | L E
tests/data/pds3/string4.lbl | L E | L E | L E | L E | L E
tests/data/pds3/tiny1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/tiny2.lbl | L E | L E | L E | L E | L E
tests/data/pds3/tiny3.lbl | L E | L E | L E | L E | L E
tests/data/pds3/tiny4.lbl | L E | L E | L E | L E | L E
tests/data/pds3/units1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/units2.lbl | L E | L E | L E | L E | L E
>
and with -v
:
> pvl_validate -v tests/data/pds3/*lbl
ERROR: PDS3 encode error tests/data/pds3/dates.lbl PDS labels should only have UTC times, but this time has a timezone: 01:12:22+07:00
ERROR: PDS3 encode error tests/data/pds3/set1.lbl The PDS only allows integers and symbols in sets: {1.5}
ERROR: PDS3 encode error tests/data/pds3/set2.lbl The PDS only allows integers and symbols in sets: {2.33, 3.4}
---------------------------------------+-----------+-----------+-----------+-----------+----------
File | PDS3 | ODL | PVL | ISIS | Omni
---------------------------------------+-----------+-----------+-----------+-----------+----------
tests/data/pds3/backslashes.lbl | L E | L E | L E | L E | L E
tests/data/pds3/based_integer1.lbl | L E | L E | L E | L E | L E
tests/data/pds3/dates.lbl | L No E | L E | L E | L E | L E
tests/data/pds3/empty.lbl | L E | L E | L E | L E | L E
[... output truncated ...]