Home | Back
New in version 0.6.0:

  License

  * The GetData library and its bindings are now distributed under the GNU
    Lesser Public License, version 2.1 or later.

  Dirfile Changes

  * GetData now support Dirfile Standards Version 7.  Standards Version 7 adds
    complex data types, a signed bitfield (SBIT), a polynomial derived field
    (POLYNOM), and other features detailed below.
  * Metafields can now be specified using a normal format specification line and
    their full (slashed) field code.
  * Field codes can now contain a "representation suffix", which allows
    specifying a complex norm used to convert complex valued data into purely
    real data.
  * The n_fields parameter in a LINCOM specification is now optional.
  * A new encoding scheme which supports LZMA compressed data using the .xz
    container format has been added.
  * Look-up tables may contain a complex y-value.  This prohibits writing to
    the associated LINTERP field.

  Library Changes

  * The field parameters spf (RAW), bitnum and numbits (BIT and SBIT) are now
    restricted to 16-bit numbers, which should be more than enough.  The public
    API uses gd_spf_t and gd_bit_t for these quantities.
  * The shift field parameter is now a 64-bit integer type, called gd_shift_t.
  * The dirfile flags, including encoding and endianness symbols are now
    explicitly long ints.  Previously these quantities mixed long int with int
    arbitrarily.  This affects the public API.
  * The parser callback has received a new, API breaking signature, which should
    hopefully reduce the amount of API breakage in the future.  It is also now
    possible for the caller to pass data through GetData to the callback, if
    desired.
  * Computation of derived fields can now handle complex valued inputs, and
    computation will occur in the complex plane when required.
  * The computation of LINCOM fields has been re-written and is now ~20% faster
    for LINCOMs with 2 or 3 input fields.  (No change for LINCOMs of 1 field.)
  * A new open flag, GD_PRETTY_PRINT, tells the library to attempt to write a
    nicer looking format file.  What this specifically means, isn't part of the
    public API and shouldn't be relied on in portable code.
  * BUG FIX: dirfile_madd_spec() and dirfile_add_spec() didn't properly flag
    the fragments they modified as dirty, leading to possible loss of metadata.
  * BUG FIX: calling get_nframes on a bzip2 encoded dirfile would report the
    wrong dirfile length.
  * BUG FIX: LINTERP tables are now closed after loading, resolving a file
    descriptor leak of great age (ie. pre-0.3.0).
  * BUG FIX: When re-writing a format file fragment containing fields with
    CONST scalar field parameters, GetData no longer replaces the CONST field
    codes in the field specification lines with the value of the CONST field.
  * BUG FIX: when moving a field with dirfile_move(), the fragment index in
    the entry objects of metafields defined for the field were not updated.
    (The metafields were placed in the correct fragment when the metadata was
    flushed to disk, despite this.)
  * BUG FIX: Calling getdata() or putdata() on 64-bit architectures in certain
    situations involving a non-zero frameoffset no longer results in bizarrity.
  * BUG FIX: A memory leak associated with RAW field metadata has been fixed.
  * BUG FIX: 64-bit integer literals in the format file are no longer truncated
    to double precision when read.

  API Changes

  * This release breaks ABI compatibility and, to a lesser extent, API
    compatibility.  The SOVERSION of the library has been incremented
    accordingly.
  * A C-89 API will be declared by getdata.h if GETDATA_C89_API is defined
    before inclusion.  If this symbol is not defined, the C API defined will
    include C-99 conforming complex types, and getdata.h will include complex.h
    if `complex' has not been defined before inclusion.
  * Due to the new, simple way of specifying META fields in Standards Version 7,
    dirfile_add_spec() and dirfile_alter_spec() can be used to modify
    metafields. dirfile_madd_spec() and dirfile_malter_spec() remain as well.
  * An array containing the CONST scalar field codes used when non-literal
    parameters are used in the specification of various different fields has
    been added to the gd_entry_t object.  As a side-effect of this,
    dirfile_add(), dirfile_alter_entry(), and dirfile_madd() can be used to
    manipulate non-literal field parameters.
  * A function, get_native_type(), now exists which returns the underlying data
    type of a specified field code.
  * A function, get_framenum(), is now available to perform a "reverse look-up"
    on a field: given a data value, it reports the frame number where the field
    has that value.  The field must be monotonic.
  * A function, dirfile_validate(), now exists which can be used to check the
    validity of a field code before use.
  * Two new error codes have been added: GD_E_DOMAIN, potentially returned by
    get_framenum(), and GD_E_BAD_REPR, indicating an unrecognised representation
    suffix, or an attempt to write to a representation.
  * A new syntax error suberror code has been added: GD_E_FORMAT_LITERAL, which
    indicates a syntax error in a complex literal number.
  * BUG FIX: On success, dirfilename(), dirfile_reference(), and get_reference()
    didn't change the error code to GD_E_OK, resulting in spurious error codes
    from these functions.
  * BUG FIX: dirfile_madd_string() and dirfile_madd_const() now don't fail
    erroneously by reporting GD_E_BAD_CODE.
  * BUG FIX: error strings relating to GD_E_DELETE error codes now report the
    proper error information.
  * BUG FIX: the list returned by get_mstrings() is now properly NULL
    terminated.
  * BUG FIX: the entry->table member now reports the proper path to the LUT.
    (That is, the path stored in the format file.)  Previously it incorrectly
    reported a path relative to the base dirfile directory.

  Legacy API Changes

  * Because they would otherwise be unreported, in the structure returned by
    GetFormat(), POLYNOM entries are reported as LINCOM entries, by discarding
    higher order terms.  Similarly, SBIT entries are reported as BIT entries.

  Bindings

  * Bindings for Python and the Interactive Data Language (IDL) have been
    added.
  * Bindings for the functionality introduced into 0.6.0 have been added.
  * C++: The missing bindings for dirfilename(), get_nvectors(), and
    get_fragment_index() have been added.
  * C++: Empty entry class constructors have been added which allow delayed
    specification of field metadata.
  * C++: The dirfile.h header now includes all other C++ headers, and is now
    the only header required to be included in applications.  (The other headers
    can still be included, with no ill effects.)
  * All bindings have a much expanded test suite, which has resulted in the
    discovery of many bugs (subsequently fixed).
  * F77: The callback function is now dirfile-specific, as it is in the C API.
  * C++ BUG FIX: Entry class constructors now properly set the field name for
    metafields.
  * C++ BUG FIX: Fragment::SetEncoding() now actually does what it advertises,
    instead of doing nothing.
  * C++ BUG FIX: The recode argument to various functions is now optional and
    defaults to zero, as the documentation already indicated.
  * C++ BUG FIX: dirfile_close() was not being called by the Dirifle destructor.
  * F77 BUG FIX: a NULL C string returned from the C API now returns an empty
    string, rather than causing a segfault.
  * F77 BUG FIX: attempting to call another F77 function before calling GDCOPN
    or GDOPEN no longer results in a segfault.
  * F77 BUG FIX: passing an out-of-range dirfile unit no longer results in
    undefined behaviour.
  * F77 BUG FIX: the last character of a string returned by the bindings is no
    longer deleted.
  * F77 BUG FIX: string lengths now report the correct length.
  * F77 BUG FIX: passing dirfile unit 0 to GDCLOS or GDDSCD no longer corrupts
    the bindings.
  * F77 BUG FIX: GDFLDN no longer accepts indices less than one.
  * F77 BUG FIX: GDESTR now space pads its output string, rather than filling
    the remainder of the string with garbage.
  * F77 BUG FIX: The get_entry() bindings, GDGExx no longer segfault on dirfile
    error.
  * F77 BUG FIX: GDGTST and GDPTST now report the FORTRAN string length, which
    is one less than the C API would report, due to the lack of a terminating
    null.
  * F95 BUG FIX: Attempting to retrieve a PHASE entry with fget_entry() now
    works.
  * F95 BUG FIX: Several routines which were named fdrifile_<foo> are now
    properly spelled.

  Miscellaneous

  * All bindings can now be disabled by passing --disable-bindings to
    ./configure.

|==============================================================================|

New in version 0.5.0:

  Dirfile Changes

  * Support for two new encoding schemes has been added which handle bzip2 and
    gzip compression.  Like the slim encoding, the bzip and gzip encoding
    schemes currently do not support writing, but do allow reading dirfile data
    compressed with the standard gzip and bzip2 utilities.  Reading compressed
    data is unsurprisingly slow.

  Library Changes

  * Encoding schemes relying on external libraries (slim, gzip, bzip2) may now
    be built into stand-alone library modules which will be loaded, as needed,
    at runtime.  GetData will fail gracefully if modules are missing.  See the
    README for full details.
  * BUG FIX: Writing metadata to disc now preserves the permissions of the
    format file fragments which are changed.
  * BUG FIX: Format files without a line feed character at the end of the file
    no longer cause the parser to segfault.
  * BUG FIX: putdata() now reports the correct number of samples written for
    ASCII encoded files.
  * BUG FIX: GetData no longer inserts unnecessary "/./" elements into the paths
    it reports.

  API Changes

  * The caller can now register a callback function with a DIRFILE by opening
    the dirfile with dirfile_cbopen instead of dirfile_open.  This callback
    function will be called by GetData whenever the format file parser
    encounters a syntax error.  The callback function can be used by the caller
    to either correct the syntax error, or else tell GetData whether to ignore
    the line or not.  The callback function may be later modified by calling
    dirfile_parser_callback.
  * A fragment may be removed from a dirfile by calling dirfile_uninclude.
  * The pathname of the dirfile may be retrieved by calling dirfilename.
  * The reference field for the dirfile may be set or retrieved by calling
    dirfile_reference.
  * Fragment metadata may be queried or modified by calling:

      - get_encoding, dirfile_alter_encoding
      - get_endianness, dirfile_alter_endianness
      - get_frameoffset, dirfile_alter_frameoffset
      - get_protection, dirfile_protect
      - get_parent_fragment

  * Various functions now exist to modify field metadata:

      - dirfile_alter_entry, dirfile_alter_<field-type>, dirfile_alter_spec,
        dirfile_malter_spec
  
  * A field may be moved to a different format file fragment using dirfile_move.
  * A field may be renamed by using dirfile_rename.
  * A field may be removed from the dirfile by calling dirfile_delete.
  * The type or fragment index of a field may be retrieved by calling
    get_entry_type or get_fragment_index, respectively.
  * The pathname of a raw field may be obtained from a call to get_raw_filename.
  * A DIRFILE may be deallocated without saving modified metadata by calling
    dirfile_discard.
  * A new flag, GD_IGNORE_DUPS, may be passed to dirfile_open to tell the
    parser to ignore duplicate field specifications in the format files.  This
    is really only useful identically duplicate specifications, since there is
    no indication of which of the duplicates is honoured.  Explicit control can
    be obtained by handling this inside a caller supplied callback function.
  
  Bindings:

  * All functions in the Fortran 77 bindings have been completely renamed from
    GDFxxx to GDxxxx to provide a larger namespace for our use.
  * Fortran 77, Fortran 95, and C++ bindings have been updated for the latest
    API changes.
  * BUG FIX: Fortran 77 functions which return character strings no longer
    corrupt memory as a side-effect of operation.  This also affected the
    Fortran 95 bindings.

  Miscellaneous:

  * BUG FIX: The dirifle_madd_bit(3) manual page has been corrected to show
    the correct order or parameters for all the dirfile_madd_<field_type>
    functions.

|==============================================================================|

New in version 0.4.2:

  Library Changes

  * BUG FIX: The default protection level of format file fragments is now
    properly set to "none".  Reported by Brendan Crill.
  * BUG FIX: getdata() now reports the correct number of samples read for slim
    and ASCII encoded files.  Reported by Adam Hincks.
  * BUG FIX: A segmentation fault only encountered when reading slim encoded
    data has been fixed in dirfile_flush().

  Legacy API Changes

  * Several fixes have been made to better re-create the legacy API:

      - the return value of GetFormat is no longer marked const (although
        it should be considered so).
      - a symbolic link getdata_struct.h -> getdata.h is created when the
        headers are installed, but only if the legacy API is present in the
        library.
      - when included with a C++ compiler, the legacy function prototypes
        are wrapped in an extern "C" block. (The new API functions are not.)

|==============================================================================|

New in version 0.4.1:
  
  Library Changes

  * BUG FIX: Adding an ASCII encoded RAW field no longer creates an empty,
    unencoded binary file.
  * BUG FIX: A few minor memory leaks have been patched.  Most of these
    were the result of insufficient clean up in dirfile_close().
  * As an exception to the Standards, the full stop character '.' is once
    again permitted in field names.  This is required by BLAST.  Instead,
    field names are prohibited from ending with the known encoding extensions.
    The Standards still prohibit the full stop character in field names, and the
    GD_PEDANTIC flag will disable this exception.  Furthermore, fields may not
    be added via GetData which contain a full stop character.
  * Some minor speed improvements have been made to the format file parser
    resulting in a ~30% decrease in parsing time.

|==============================================================================|

New in version 0.4.0:
  
  Dirfile Changes

  * GetData now supports Dirfile Standards Version 6.  Standards Version 6 adds
    numerical (CONST) and string (STRING) scalar fields to the dirfile, as well
    as other features outlined below.
  * CONST names can now appear as parameters in field specifications in certain
    places in lieu of a literal number.
  * The tokeniser has been re-written.  Any sequence of non-NUL bytes can now
    form a valid token.  Field codes may not have ASCII control characters
    or the "reserved characters" &, /, ;, <, >, |, . but may contain any other
    characters (including whitespace).
  * Subordinate fields may now be attached to fields.  These subfields are
    defined with the META directive, and referenced in most cases as
    "<parent-field>/<subfield>".
  * Binary files in dirfiles may now be "encoded".  In addition to raw
    (unencoded) files, two encodings are currently supported:

      - ASCII encoding (a simple proof-of-functionality encoding), and
      - Slimlib encoding (a compression library used by ACT).

    Encoding schemes are fragment-local, although they are inherited from parent
    fragments.
  * Endianness is now also fragment-local, meaning different endiannesses can
    co-exist in the same dirfile (so long as the fields are defined in different
    format file fragments), and GetData will do the Right Thing.
  * "FILEFRAM" is no-longer supported as an alias for "INDEX".  This affects
    GetData's implementation of all Standards Versions.  Code that made use of
    FILEFRAM will have to be updated to use INDEX instead.
  * The reference field, previously the first RAW field in the format file,
    may now be specified using the REFERENCE directive.  The reference field
    is the field that is looked at when get_nframes() is called.  This is
    important to programs like kst which support streaming dirfiles.
  * Portions (or all) of a dirfile's data and metadata can be protected from
    change by specifying the PROTECT directive.  This protection is "advisory",
    ie. while GetData will respect it in all cases, don't count on the PROTECT
    directive to ensure data integrity.

  Library Changes

  * INDEX is now a normal field.  It appears in the count/list functions.
    Attempts to write to INDEX will fail.
  * The GD_E_EMPTY error code has been removed: it is no longer an error to
    query a dirfile containing no RAW fields.
  * The presence of two fields in the dirfile with the same name is now
    detected, and will cause a syntax error when parsing the format file.
    Previously, such things were allowed, but only one of the synonym fields
    could ever be queried. (Which field was returned was arbitrary.)
  * Field code look-ups for input fields are now cached, which should result
    in slightly better performance from the library.

  API Changes

  * Fields may now be added to dirfiles in a number of different ways:

    - by passing an gd_entry_t to dirfile_add()
    - by passing a field specification line to dirfile_add_spec()
    - by passing field parameters to one of the dirfile_add_<foo>() functions.

  * Whole other format file fragments may also be added to the dirfile by
    calling dirfile_include().
  * A function is now present (dirfile_metaflush()) to flush metadata changes to
    disk (by re-writing format file fragments).  dirfile_flush() and
    dirfile_close() will also flush metadata, if needed.
  * STRING and CONST values can be retrieved/set by calling (get,put)_string()
    and (get,put)_constant().  (get,put)data() are only for vector type fields.
  * META fields can be queried/set like normal fields using the get/put
    functions and their full (slashed) field code.
  * In addition to get_nfields() and get_field_list(), there are now
    corresponding functions that provide lists/counts of vectors
    (get_vector_list()/get_nvectors()), particular field types
    (get_field_list_by_type()/get_nfields_by_type()), as well as functions
    that provide lists of string values (get_string_values()) and constant
    values (get_const_values()).
  * Analogous functions for the add, list, and counting functions exist for
    META fields, as well.
  * Extra flags have been added to dirfile_open() to permit indicating the
    encoding type, if not specified in the dirfile itself.
  * DIRFILE struct members and gd_entry_t private members are now completely
    hidden from the public interface.  Where previously callers could query
    dirfile->error to check for an error, they must now call get_error(dirfile).
    This change was made to reduce unintentional ABI breakage when modifying
    internal library properties.
  * GD_VERBOSE has been added to the list of available dirfile_open() flags.
    If this flag is specified, GetData will write errors to stderr as
    encountered.

  Legacy API Changes

  * Error codes which cannot be returned by the legacy API might not have a
    corresponding string in GD_ERROR_CODES[].  Instead, these entries will
    simply be the NULL pointer.
  * No facilities exist in the legacy API to set or query CONST or STRING
    fields.  However, META fields of vector type can be queried/set using
    GetData() and PutData(), as in the new API.

|==============================================================================|

New in version 0.3.1:

  Legacy API Changes

  * BUG FIX: Dirfiles are now opened in read-only mode, unless instantiated via
    PutData(), allowing GetData() calls on read-only dirfiles.  If
    PutData() is called on a dirfile previously opened read-only, it will be
    re-opened in read-write mode.

  Bindings

  * The C++ bindings, formerly called libdirfile, are now called libgetdata++
    to be more explicit about what this library is.

  Miscellaneous

  * The package now includes pkg-config support for libgetdata.

|==============================================================================|

New in version 0.3.0:

  Dirfile Changes

  * GetData now supports Dirfile Standards Version 5 which includes support
    for signed 8-bit, and signed and unsigned 64-bit integer types.
  * As part of Standards Version 5, the restriction on field name length has
    been removed.  (The filesystem will impose an effective limit on RAW fields
    of a few hundred characters, and format file lines are limited to several
    thousand, limiting derived field names.)
  * The library can now convert between big and little endiannesses.  An
    optional directive, ENDIAN, added in Standards Version 5, is available to
    specify the byte-sex of a dirfile.

  API Changes

  * There is a new interface which fixes issues with thread safety and largefile
    support in the old interface.  The old interface (referred to as the "legacy
    API") is still supported, but doesn't fully implement Dirfile Standards
    Version 5.  See the README for full details.
  * putdata now respects FRAMEOFFSET.
  * putdata can now write to PHASE fields, and multi-bit BIT fields.
  * Some error codes have been renamed, and others removed or added, in order to
    regularise error codes between getdata and putdata in the sundry versions.
    Furthermore, the underlying values of some of these codes have changed
    (notably excluding GD_E_OK, which is guaranteed to be zero). Changes
    include:

      - GD_E_OPEN_FORMAT is now called GD_E_OPEN
      - GD_E_BAD_RETURN_TYPE is now called GD_E_BAD_TYPE
      - GD_E_NO_RAW_FIELDS is now called GD_E_EMPTY
      - PD_E_MULT_LINCOM is now called GD_E_BAD_PUT_FIELD
      - GD_E_OPEN_RAWFIELD and PD_E_OPEN RAWFIELD are now both represented by
          GD_E_RAW_IO
      - PD_E_BAD_CODE is now handled by GD_E_BAD_CODE
      - GD_E_FIELD, GD_E_SIZE_MISMATCH, ENDIAN_ERROR, CLOSE_ERROR are no longer
        applicable and have been removed
      - PD_E_CLOSE_RDONLY, PD_E_WRITE_LOCK, PD_E_FLOCK_ALLOC, which were were
        defined in the header but never used, have been removed
      - GD_E_TRUNC, GD_E_CREAT, GD_E_BAD_DIRFILE, GD_E_RANGE and GD_E_ACCMODE
        are new

    This affects the legacy API.
  
  Legacy API Changes

  * The legacy API has been marked deprecated.
  * Error codes have changed per the description above.
  * Only the public members of FormatType are now initialised by GetFormat.

  Bindings

  * Added bindings for C++ (libdirfile), Fortran 77 (libfgetdata), and
    Fortran 95 (libf95getdata)

  Miscellaneous

  * A rudimentary, but thorough, test-suite has been made.