During the January 12th Technical Steering Committee meeting, Kip Twitchell provided an update on our proof of concept work with GenevaERS and Spark.
Recording from the November OPM Webinar
GenevaERS TSC Chair Kip Twitchell and Solution Architect Sandy Peresie present at the monthly Open Mainframe Project
Announcement – GenevaERS Webinar
November 18th 11 AM US Eastern time
Join us to learn about GenevaERS Open Mainframe Project. GenevaERS, a single-pass optimization engine driving the data supply chain executing on z/OS.
GenevaERS offers businesses a high-level reporting solution uniquely tuned for big data scanning and improved financial transparency for better decision-making.
This project combines the processing power of a single pass of large volumes of data, an optimized join feature, the reliability of the mainframe and the dynamic open source community.
As a recent addition to the Open Mainframe Project, there are many ways to get involved and the team would love to have your help.
Open Mainframe Project Mini-Summit Europe
Kip provided an overview of the GenevaERS project at the mini summit held Friday October 30th 2020.
GenevaERS Goes Open Source
Swapnil Bhartiya, founder and Editor-in-Chief of TFiR, interviews Kip Twitchell about GenevaERS
Lucky #13 – the GenevaERS Proposal
In the summer of 2020, Kip Twitchell presented SAFR / GenevaERS to the Linux Foundation’s Open Mainframe Project’s Technical Advisory Committee (TAC).
GenevaERS Intro
Learn about our first phase of work; a proof of concept utilizing Apache Spark
Learn more about our proposed architectural direction for Geneva
Recordings and minutes from the Technical Steering Committee meeting on September 8th 2020 are now available
All recordings and minutes are stored in the tsc – meetings folder out on GenevaERS community GitHub
Direct links for the 8th are:
Sign-up and Commit to the Spark-GenevaERS POC
Organization of the project continues, but much progress has been made. Check out the Community Repository on GitHub, and its Governance and Technical Steering Committee Checklist to see what’s been happening.
But don’t stop there….
Get Connected
The first Technical Steering Committee (TSC) Meeting, open to all, is scheduled for Tuesday, August 11, 2020 at 8 PM US CDT, Wednesday August 12, 10 AM WAST/HK time.
Project Email List: To join the meeting or to be kept up to date on project announcements, join the GenevaERS Email List. You’ll receive an invitation as part of the calendar system. You must be on the e-mail list to join the meeting.
Slack Channel: Conversations are starting to happen on the Open Mainframe Project Slack workspace. Request access here, then when granted look for the GenevaERS Channel.
Spark POC Plans
The team continues to prepare the code for release. But that’s no impediment to our open source progress.
We’ve determined that the best way to help people understand the power of The Single Pass Optimization Engine is to contrast it–and begin to integrate it–with the better known Apache Spark.
Here’s what we plan to do work on over the next several weeks, targeting completion before the inaugural Open Mainframe Summit, September 16 – 17, 2020.
This POC was designed using these ideas for the next version of GenevaERS.

The POC will be composed of various runs of Spark on z/OS using GenevaERS components in some, and ultimately full native GenevaERS.

The configurations run include the following:
- The initial execution to produce the required outputs will be native Spark, on an open platform.
- Spark on z/OS, utilizing JZOS as the IO engine. JZOS is a set of Java Utilities to access z/OS facilities. They are C++ routines having Java wrappers, allowing them to be included easily in Spark. Execution of this process will all be on z/OS.
- The first set of code planned for release for GenevaERS is a set of utilities that perform GenevaERS encapsulated functions, such as GenevaERS overlapped BSAM IO, and the unique GenevaERS join algorithm. As part of this POC, we’ll encapsulate these modules, written in z/OS Assembler, in Java wrappers, to allow them to be called by Spark.
- If time permits, we’ll switch out the standard Spark join processes for the GenevaERS utility module.
- The last test is native GenevaERS execution.
The results of this POC will start to contribute to this type of architectural choice framework, which will highlight where GenevaERS’s single pass optimization can enhance Apache Spark’s formidable capabilities.

Note that the POC scope will not test the efficiencies of the Map and Reduce functions directly, but will provide the basis upon which that work can be done.
Start Contributing
Open source is all about people making contributions. Expertise is needed in all types off efforts to carry of the POC.
- Data preparation. We are working to build a data set that can be used on either z/OS or open systems to provide a fair comparison for either platform, but with enough volume to test some performance. Work here includes scripting, data analysis, and cross platform capabilities and design.
- Spark. We want some good Spark code, that uses the power of that system, and makes the POC give real world results. Expertise needed includes writing Spark functions, data design, and turning.
- Java JNI. Java Native Interface is the means by which one calls other language routines under Java, and thus under Spark. Assistance can be used in helping to encapsulate the GenevaERS utility function, GVBUR20 to perform fast IO for our test.
- GenevaERS. The configuration we create we hope to be able to extract as GenevaERS VDP XML, and provide it as a download for initial installation testing. A similar goal with the sample JCL that will be provided. GenevaERS expertise in these fields is needed.
- Documentation, Repository Work, and on and on and on. At the end of drafting this blog entry, facing the distinct chance it will be released with typos and other problems, we recognize we could use help in many more areas.
The focus for our work is this Spark-POC repository. Clone, fork, edit, commit, create pull request, repeat.
On a daily basis there is an open scrum call on these topics held at this virtual meeting link at 4:00 PM US CDT. This call is open to anyone to join.
Contributions from all are welcome.
Lucky 13: GenevaERS is an Official Open Mainframe Project project
On Thursday July 9th, GenevaERS was approved an Incubation Project under the Linux Foundation’s Open Mainframe Project.
Comments from the approving Technical Advisory Committee included (paraphrased):
- “This project showcases the types of applications that Z is very important to.”
- “Would like to see capabilities to cleanse data before feeding into Spark for machine learning.”
- “Yes, intersecting with Apache Spark would be an interesting development.”
- “This is exactly the kind of project we want to see”
- “Most of the projects we are sponsoring are user interface related, supporting modernization. This has interesting transaction processing characteristics.”
- “Looks like you already have a couple of people interested in participating/contributing…” The energy in the committee is “because this is a mature/sophisticated project.”
The following slides were used in the application discussion:
Additionally, the relationship of GenevaERS to Apache Spark was discussed, leveraging the content from the recent Proposed GenevaERS 5.0 Architectural Direction.