Custom Data Filtering
A proven and effective way to manage clean file and data transfers is through data filtering but it requires a deep understanding of the specific file structure and contents. Building custom filters can be very time consuming and require specific expertise to ensure they can be certified as required by various authorities. However, new approaches such as using the Data Format Description Language (DFDL) standard and the open source Daffodil processor for DFDL to create custom filters without having to go through time consuming and expensive re-certification process
Apache Daffodil is an open source implementation of the DFDL (Data Format Description Language) specification that uses DFDL schemas to parse fixed format data into an infoset, which is most commonly represented as either XML or JSON. This allows the use of approved well-established XML or JSON technologies and libraries to consume, inspect, and manipulate fixed format data in existing data filtering solutions. Daffodil is also capable of the reverse by serializing or “unparsing” an XML or JSON infoset back to the original data format.
To improve robustness and cost-effectiveness of Cross Domain Solutions (CDS), modern filtering designs are incorporating:
Data Format Description Language (DFDL)
Replace software for data parsing with a declarative approach that uses a DFDL schema that describes the data format, along with the Daffodil open source DFDL processor to parse data, revealing it as XML (or JSON) for inspection, validation, and sanitization. Daffodil is used for subsequent ‘unparse’ back to the native format.
Data Transformation and Filtering
Use an XML-centric approach to data inspection, validation, and sanitization.
Use XML Schema and Schematron for data validation.
Use XSLT or XQuery-based transformations on XML data.
Augment proprietary product filter sets with filters built to standard APIs from the Filter Componentization Effort (FCE) 1.0.
Owl provides services to help the CDS community with incorporation of these technologies including:
- DFDL Schema authoring services (For industry/military standard data formats and for your project specific data formats
- Consulting on DFDL schema designs that you develop in-house
- Hands-on training on DFDL, and on use of XML technologies for filtering
- Filter development and test services
- Daffodil open source software enhancement and support services
CDS systems must be highly robust yet must also incorporate innovations to further the state-of-the-art. Large open source software libraries like Daffodil enable innovative approaches but introduce a large body of new software into the CDS. Open source means your engineers could self-support, but the reality of a large and unfamiliar code base adds risks that are particularly troubling in CDS applications.
Owl engineers have been the core contributors to Daffodil, taking it from its genesis to its status today as Apache Daffodil (Incubating). See https://daffodil.apache.org/releases/2.1.0/ for more information.
Apache, Apache Daffodil, and Daffodil are trademarks of The Apache Software Foundation.
Owl serves a diverse set of markets – so let us know your business needs and one of our cybersecurity experts will be in touch with you shortly. In order to put you in touch with the right specialist on our team, please provide us with as much information as possible. Talk soon!