Introduction to z/OS

IBM z/OS is the flag ship operating system of the zEnterprise Mainframe platform. The original IBM operating systems, e.g., OS 360, OS390, etc. were designed for batch processing, had limited Internet applications and connectivity.  The IBM mainframe has always been the IT industry's leading platform for transaction processing, consolidated and secure data serving, and for supporting enterprise-wide applications. There are NO substitutes.

In the late 2001, zEnterprise and the 64-bit z/OS operating system extended their dominance of the large-scale market place.  z/OS supports TCP/IP, any Internet application subsystems, e.g., HTTP, FTP, SSH,  etc., most major programming languages, e.g., Java, C++, etc., data bases. e.g., Oracle, MySQL, etc. and directly supports UNIX and the UNIX file system.  All new zEnterprise features are fully compatible to the 50-year legacy heritage of the mainframe as well as external hardware through zLinux.  There is no other architecture or operating system that does it all.

The following diagram displays a summarized view of z/OS operating system and subsystems.    At the top of the diagram is the host operating system, i.e., z/OS, The subsystems and interfaces listed in the far right columns are normally used by application and system programmers.  The subsystems and interfaces listed in the far left column are normally used by customers, users, and other programs. The subsystems and interfaces list in the middle column provide additionally functionality to z/OS in areas such as database management, security and communications.




The z/OS operating system is a collection of base control modules (programs) that provide functions similar to any other operating system, e.g., MS-DOS, Windows, Linux, Mac OS, Android, etc. There are two differences between z/OS and the other listed operating systems.   IBM operating systems have been providing these features for over 50 years.  MS-DOS and Windows 1.0 were introduced in 1980s thirty years later.  [A history of Windows -  http://windows.microsoft.com/en-us/windows/history#T1=era0] For example, who invented virtual memory operating systems and files systems? Answer IBM.  The other operating systems modified these concepts later.  Who invented System and Server Virtualization? VMware - no.  IBM. And the list goes on. The majority of concepts that an IS or CS major learns in a class started with IBM.

IBM is no longer the major IT company. Apple is. Why? Apple is a consumer-based IT company which is much more interesting and fun and a larger potential sales market.  IBM is a very large-scale, business transaction processing company directing its services to large businesses, government agencies and other organizations.

If the data processing requirement is large-scale and security is a concern, the chances are it is IBM and z/OS.  Did you text today?  Yes, I did I text on my Verizon phone. O.K. What platform does Verizon use to process and route text messages? Yes, 95% of all text messages are processed by an IBM mainframe. One-third of the world wealth is processed and stored on an IBM mainframe. 71% of all banking, insurance and financial service transactions worldwide are processed by an IBM mainframe, 51% of retailing and 46% of government transactions.

Do these companies and government agencies love IBM?   Hell, no. There is no alternative. The past and the future disaster O' Bama Health Care website is a proof of this fact. First, let determine the functionality of the OBC website.  It simply will add (insert) individuals to a database based on some parameters and then routes the registered OBC user to sign-up for health care at a third-party web site.  The OBC does not even process an insurance sign-up transaction.  It doesn't even perform even the most simplest transaction of Amazon.com.  [Which Enterprise architecture does Amazon use? Do you even have to ask?]

The reporting transaction processing bottle neck at OBC the web site starts with the fact that it cannot or ever will support more than 300,000 logon users and transactions per day.  [Can you imagine Amazon with this limitation?]  JP Morgan Chase is the home of VISA credit card transaction processing. The zEnterprise EC 12 mainframe running z/OS process 3 million logon transactions per minute.

The following table core concepts provided my most operating systems like z/OS. The next table will summarize the roles of popular add-on z/OS subsystems.


z/OS Core Functions	Description and Comparison
Task  and Time Management 	An operating system task, or processes, is an executing program under the control of the operating system. An operating system will execute these tasks using different processing modes or workloads, e.g., batch, time sharing, etc.  In a multitasking operating system, one must trade off efficiency (throughput) with response time.  Batch processing is very efficient, but has poor response time when multi-users at waiting. (See Processing Modes later.)
Memory Management	All programs and data must be physical memory to be executed.  Physical memory is very fast and expensive, and its total size is limited. Most modern operating system using virtual memory. A  VM OS will temporarily store pages of executing programs and data on a disk, and swapped back to main memory as needed. VM OS have the ability to manage the execution of large amounts of tasks and data by using an auxiliary storage device as an overflow.
I/O Control 	Auxiliary storage systems, network devices and other computer peripherals are connected to CPU and system memory and software must be used to control the communication between these components.  Most are familiar with the concept of "drivers" used by the Windows operating systems.

In very early versions of IBM OSes, the I/O control and communication programs where compiled into the operating systems.  This compilation method provided the maximum efficiency for I/O communication, but made it difficult to modify the I/O Configuration. Legacy IBM I/O control was based on device type, channels and unit numbers.

In the last couple of decades IBM used the same popular I/O interfaces and I/O driver model as used by microcomputer operating systems.
File Systems	A files system is a collection of modules that controls how data is stored and retrieved in a meaningful way.  The disk I/O control system will store data in the format of cylinders, tracks, and perhaps sectors (Windows, Linux and Unix use sectors).  But, that format is not meaningful.

The basic unit of storage for the z/FS is the data set. A data set is similar to file in Windows. The content of a data set is very primitive; whereas, the content of a Windows file varies considerably, e.g., *.DOCX, MP4, *.MDBx, *.HTML , etc.

The methods that a file system uses to organize files vary considerably. The Windows NTFSv5 file system is a hierarchical directory structure. z/FS uses a VTOC table which is similar to the root directory of NTFS5, and a Partitioned Data Set (PDS) that is similar to a subdirectory or folder.  (See Introduction to File Systems Section Later for more details.)
File System Integration 	The z/FS was made for efficient file storage and processing, not flexibility. The Windows NTFSv5 and Linux ext-4 file systems were designed for the flexibility of modern desktop and mobile operating system. To provide the best of both worlds z/OS does support UNIX and the UNIX file systems.  Linux does support multiple operating systems, but not on the same data sets or files. The beauty of zEnterprise is that one view and process data sets and files by using either the z/FS for legacy applications and the Unix file systems on the same storage device.






Major z/OS Subsystems



Major z/OS Subsystems 	Description and Comparison
TSO/E 	TSO is both a user interface and a processing mode.  The only way system and application programs were able to communicate with an IBM operating system was through punched cards. There were no user-friendly interfaces and remote processing was very difficult.

TSO provide a command-line interface in which key
ISPF
JES3
SDSF
CICS
z/OS HTTP Web Server
WebSphere
WebSphere MQ
DB2
IMS
RACF
z/OS Communication Server and SNA
RACF
zEnterprise FTP Server



  been around for 26 The following diagram displays a summarized view of z/OS operating system and subsystems.    At the top of the diagram is the host operating system, i.e., z/OS, The subsystems and interfaces listed in the far right columns are normally used by application and system programmers



Introduction to z/OS Workloads or Processing Modes

IBM supports four types of processing modes to support application and system programs: batch processing, CICS, WebSphere, and WebSphere MQ.  IBM refers to each types of processing modes as "Work Loads".


Batch Processing

The original and most efficient processing mode is called Batch Processing. After an application has been developed, a batch application is submitted to the Job Entry System (JES3) using JCL for immediate or delayed (hold queue) execution.  Once the JCL is processed by JES, it will review the availability of required resources, e.g., processing time, RAM, input and output data sets, etc., before the job is actually started.

While many consumer applications have shifted to web and mobile applications that process real time transactions, the use of batch processing continues to evolve. Local, state and the federal government (especially the IRS and Social Security administration) use batch processing to processes taxes, benefits, and billing. While payroll systems may enter data online, payroll processing applications are normally executed using batch processing. Modern data warehouses transfer transaction data from transaction databases using a batch procedure called extraction, transfer and load (ETL), and data mining and other computation processing relies on batch processing.

Advantages of Batch Processing

	Requires no user interaction; hence does not wait for user input of data or initiation of job execution. Thus, it avoids idling of computing resources with minute-by-minute human interaction and supervision.
	Batch jobs can be scheduled to take advantage of idle time and load balancing requirements.
	Minimizes idle time of required resources, e.g., input data sets, waiting output devices.  High utilization rate.
	With proper scheduling it permits resources to be shared more efficiently, assign priorities, and with less conflicts.
	Generally preferred for heavy computational processing, significant update transaction workloads, long running jobs to be executed on multiple nodes (clusters or grid computing), e.g., IBM SysPlex and Parallel SysPlex infrastructure

Disadvantages of Batch Processing
	No uses interaction or support for Online Transaction Processing (OLTP), e.g., transaction and updates entered by users are processed immediate.
	No web or mobile interfaces.
	Limited sharing of datasets

Customer Information Control System (CICS)

Online Transaction Processing (OLTP) is a processing mode that users normally enter data for a small transaction and the system is updated immediately.  For example, a batch program may calculate interest charge or update payments for all customer accounts based on inputs from a dataset. But, OLTP will permit users and employees to enter data on form, one-customer-at-time,  and then immediately update the customer's balance one-customer at a time.

IBM CICS, a high volume OLTP system, was introduced in 1965. For decades, there were not viable commercial alternatives.  OLTP and CICS systems have two major components: an online transaction monitor and an application programming interface (API).

Any operating system has very limited capabilities to share applications and data between users. For example,  Windows XP introduced multithreading in Microsoft's desktop operating systems. Multithreading is the ability of an operating system to execute and share the same copy of an executable module, e.g., Internet Explorer, using the same process id (task number).  Multitasking can execute two or more separate copies of  an executable module using a separate task number and duplication of resources.   Batch processing uses multitasking, but  more important cannot concurrently share data sets.

CICS,a transaction monitor, provides  multithreading services and  the ability to share VSAM datasets in a read-write mode. DB2 and IMS will provide the abilities to share tables, but not VSAM data sets.  CICS may be compared to modern web application servers, e.g., Apache Tomcat (free open source) and Web Sphere, Apache Geromino, .NET framework, JBOSS (proprietary)   All provide similar advantages:  share ability of code and data, code and data integrity, centralized management, security, fault tolerance, load balancing, performance, and transaction  recovery services

However, CICS has been providing transaction services since 1965 (not the 1990s),  and provides scalability, security, and performance levels that that modern web application servers cannot come close.  There are twice as many CICS transactions processed in a day that the number of web pages served.  The foundation of CICS's dominance is CICS is optimized to take advantage of zEnterprise servers and z/OS.  But, of course CICS is not either free or inexpensive.

Traditionally, CICS interfaces with users via a CICS map. A CICS map is a text-based user interface similar in functionally to a HTML web page. CICS maps can be displayed on dumb terminals or terminal emulators. CICS maps requires little processing on the client side. On the other hand, a HTML web page is a complex, multimedia user interface with requires significant processing at the client side. Between 1960 and 1980 there were very few microcomputers. CICS was the only alternative.

The second component of CICS is the appliance programming interface (API).  COBOL was originally designed for batch processing.  CICS and application services sits between the host operating system and COBOL executable module.   In a CICS infrastructure, z/OS executes one or more copies (regions) of  CICS.  CICS in turn executes one or more COBOL applications.  In program terminology, CICS calls a COBOL load module, which in turn the COBOL load module calls (requests) services from other CICS  programs.  COBOL does not change. But, the application programmer adds statements to the COBOL program contains identified  a CICS API block, e.g., EXEC CICS …. CICS API Statements … END-EXEC.  Before the COBOL program is compiled these CICS API block are placed by traditional COBOL statements.

Interfacing with CICS is a very complicated coding process.  To make the life of a COBOL programmer easier, CICS provides these easy-to-code CICS API blocks. CICS provides APIs for PL1 and Java.  All Java web application servers are based on Java, which also includes to the Java Enterprise APIs.  While J2EE APIs provides similar functionality to COBOL CICS APIs, both COBOL and CICS APIs are much easier to learn and code.

Advantages of CICS (and many other OLTP systems)
	Immediate execution of a transaction.  Users can enter transactions and the system may be updated immediately.  Some CICS transactions just stores data in a batch transaction file to be processed later, e.g., a commercial bank in which employees enters checks enter to file and the transactions to update the balances will be processed later.  Online processing is difference than real time processing.
	Provides services not provided by the host operating system, e.g., share ability of code and data, code and data integrity, centralized management, security, fault tolerance, load balancing, performance, and transaction  recovery services.
	CICS provides APIs for several programming languages, J2EE does not.
	While originally designed to support 3270 terminals over a SNA network, CICS now supports TCP/IP, web pages, mobile applications, and web services

Disadvantages of CICS (and many other OLTP systems)
	Cannot to be scheduled and generally requires resources 24/7.  Idle time
	Extra services require extra processing. The cost per online transactions is substantially more expensive than the cost per batch transaction.
	Increase application programming complexity, e.g., user interfaces, user validations, help screens, navigation, etc.
	Complicate configurations are required to balance peak loads and system utilization objectives



IBM WebSphere


IBM WebSphere is IBM's proprietary version of a J2EE server, which provides a web application user interface, i.e., a webpage.  Unlike other J2EE web application servers, WebSphere can be executed WebSphere can take advantage of zEnterprise operating systems (z/OS Unix and zLinux), hardware and middleware systems.  For example, assume that an organization supports COBOL batch applications, VSAM data sets, DB2 databases, and traditional CICS Maps. WebSphere can permit a web application share the same VSAM dataset or DB2 database as a traditional COBOL batch application. This provides an infrastructure design that supports the existing investment while providing the advantages of extending J2EE functionality.

Advantages of IBM WebSphere

	Provides a J2EE web interface and application server that supports and integrates with the existing investment in zEnterprise.
	Takes advantage in the pool and experience of J2EE programmers
	Provides scalability, security and performance not available to other proprietary J2EE solutions.
	Can interface with CICS

Disadvantages of IBM WebSphere
	Based on the Java program language as used in other J2EE platforms, IBM WebSphere Java applications execute 10-40% slower than COBOL CICS load modules.
	Based on the J2EE application server as used in J2EE platforms, a IBM WebSphere Application server executes 10-20% slower than a CICS transaction monitor.
	Any J2EE platform, including WebSphere, cannot scale or perform as efficiently as CICS

IBM WebSphere MQ

Application programs have historically been able to call (or invoke) another separate application load module and send/receive data. However, these applications must be supported by a common programming language, operating system and methods of communications between distributed systems. While there were several proprietary of program language solutions, today's modern application distributed systems support a collection of web services or REST-full protocols,  e.g., XML, SOAP, WSDL, HTTP, JSON

Both WebSphere and CICS were traditionally designed to support remote user-to-application solutions. WebSphere MQ will add a second specialized application server that will support messages sent between applications (not users). Applications based on different programming languages and operating systems can easily pass real-time messages.

Remote distributed applications are not directly linked, rather the share data must be process by a web service application server on both the client and server side. The calling application sends the message to client web service server which stores the message in a queue (memory or data set storage areae). The sending sever may transmit the message immediately (synchronous) or later (asynchronous). Likewise, the server side can process the message immediately or later.

MQ stands for Message Queing.  Messages represent the data shared between remote applications. Queues represent the waiting area, e.g., memory or data sets, used to control transaction flow, sequencing and timing of execution, fault tolerance and transaction recovery.  Other web service solution includes Apache Axis, WSO2, JBOSS, Oracle Application Server, etc.,

Web applications require a user interface to start a transaction. MQ and web services are started by another application. Web Services is the foundation of supply-chain and business partner ERP services.  Companies like BNY-Mellon expect that 50-70% of new application development will use web services.

Introduction to File Systems

File systems - A file system is a collection of system programs that control the manner in which files are named, are created, and data is stored and later retrieved. Each operating system supports one or more file systems.

	Consider the Windows file system named NTFSv5.  To name a Window's file one may use up to 255 characters, include embedded spaces, and will retain the case of letters in a file name.   The difference between upper and lower-case letters in NTFSv5 does not make the file name unique, e.g., PACKY.DOCX is the same file name as packy.docx.

	Files and z/OS data sets are containers that may store records. Records contain structured information such as the employee name, address, and phone number of an employee information file. Files and z/OS records may also contain unstructured information as a collection bytes; such as graphic, audio and video files.

	File systems specify rules for creating a file or data set. In Windows NTFSv5, a file is automatically created when you select SAVE for the first time. When created a file must be named, a file type must be assigned, a stored location is selected, storage space is allocated to the container, and the data content is then stored in the file container, e.g., the content of a Word document is stored in a file of the type  named .DOCX.

	The file system does not understand the details of the content of a file or data set. Remember a file is a container. The applications, e.g., Word, a COBOL Program, etc. that store the data in the file container will understand the structure of the content, not the operating system or file system.

	Every file system has structure that organizes files and improves the efficiency to find and locate files. Most modern hard disk or solid state drives (DASD) may individual store billions of files and the files system may organize files across thousands of distributed hard drives. I can't remember which drawer I put (stored) my socks.  Can you imagine the complexity of organizing files in today's file systems.

	Windows, UNIX and Linux organize files by storing them in a file folder or directory. A file folder or directory is similar to the old-fashion card catalog of a library. A library catalog contains the name of book and provides information concerning the location of the book in the library or located at other libraries. Today we simple use Google to find a location of the restaurant. Windows, UNIX, and Linux use a hierarchical file system. Within a hierarchical file system a file folder may contain a subfolder, and a subfolder will contain lower-level subfolder.

	IBM's z/FS is a flat file system, and it is not hierarchical. The closet thing that z/OS has to a folder is a Partition Data Set (PDS) - to be discussed next.  While a z/FS PDS organizes and contains members, a PDS dataset can only contain sequential members, but cannot contain another PDS.  Hierarchical files systems can organize files at multiple levels, but a flat file system can at best organize files (data sets) at one level.

	While a hierarchical file system can better organize a large number of files better, the performance to locate a file is slower than the z/FS. When z/OS needs to organize a large number of files one can use the zEnterprise UNIX file systems. The z/FS and UNIX files systems can be interchangeable depending on the storage objectives, e.g., performance versus organization.
	File systems frequently contain file security data stored in folders/subdirectories or the properties of file.   The z/FS does not directly contain security data. RACF, the z/OS security server, will store data set security data based on the data set name, e.g., the HLQ.





Data Sets and Data Set Organization

Data sets - In z/OS, a data set is a named collection of related data records that are retrieved and stored by an assigned name.  A data set is equivalent to a file as used in the Windows, UNIX and Linux file systems.

A data set is a container. The manner in which z/OS structures the internal content of the data set is called the data set organization. JCL uses the attribute DSORG to specify the data set organization.  There are two popular types of z/OS data set organization: Sequential Data Sets (PS organization) and Partitioned Data Set (PO organization).


Sequential Data Sets (PS organization)   - A sequential data set is a collection of records written and read in sequential order from beginning to end.



In z/OS, a sequential data set will normally contain data to be processed by a program, e.g., an customer data set.  Each record will contain one customer. The above information is displayed for sequential data set named RMUI001 DATA.ACCOUNTS. Each student has a copy of sequential data set. Each customer record is 170 characters long.  The first 72 characters of each customer record is displayed below.


Partitioned Data Set (PO organization) -  A data set that contains a directory and sequential members.

When you create a sequential dataset (PS), it occupies a large amount of storage space (minimum 50,000 bytes). Sometimes for doing trivial tasks, storing only a few records, you would not be utilizing the entire space available in a sequential file. Thus most of the space in the Sequential file is wasted and is blocked.

IBM provides a way to split-up or partition the space in a sequential data-set. IBM Software engineers invented the Partitioned Dataset, often called PDS. Each part is called a member. A Partitioned Dataset can have many members (parts). Each member behaves like a sequential file on its own.

How does the PDS then keep track of its members? A Partitioned Dataset (PDS) includes a directory Similar to Windows and Linux. The members of a PDS can be scattered hap hazardly anywhere in the vast computer storage space. Well-then, you consult the directory to find out the members in the PDS. Just as in a book, the index tells you, the keyword you are looking for and the page no. where it can be found, the same way, the directory tells you the member name and the computer memory address where you can find it. Thus, the directory maintains the list of member-names and pointers to the actual physical place in computer memory, where they are stored.

Not just this, but the directory also keeps track of its statistics, when the member was created, when was it last modified/viewed by somebody, how much space it occupies, the TSO user-id that created it amongst other things




On the Windows Operating System, we generally keep related files together in a folder. On the same lines, you keep related members together in one PDS. What a folder is to Windows OS, a Partitioned dataset (PDS) is to z/OS. Folders contain many related files. A PDS contains many related members.