Contents
1.0 System Memory Architecture  and Memory Failures	4
1.1 Mom, I Learning How to Program	5
1.2 Overview of RAM Technologies	6
1.3 Memory Arctitecures:  Static Versus Dynamic	8
1.3.1 SRAM (static RAM)	8
1.3.2 DRAM (dynamic RAM)	8
1.3.3 SDRAM (dynamic RAM)  and DDR	8
1.4 SIMM versus DIMM Form Factors	9
1.5 Dual Channeling	9
1.6 Dual Channel Versus Double-Sided Memory	10
1.7 Error Checking: Parity	10
1.8 Error Checking: ECC	10
1.9 Limitations of Error Checking and Error Checking Diagnostic Tools	11
1.10 IBM z System Predictive failure analysis and S.M.A.R.T.	11
1.11 Symptoms of Memory Failures	12
1.12 What Causes Memory Damage?	12
1.13 How to Test Your Windows Computer’s RAM for Problems	13
1.13.1 Windows Memory Diagnostic Tool	13
1.13.2 CPU-Z  Tool	14
1.14 Memory Speeds	14
1.4.1 Memory Speed: Nanosecond Ratings	14
Memory Speed: MHz Ratings	14
1.14.2 Memory Speed: PC and PC2 Ratings	15
1.14.3 Memory Speed: CL and RL Ratings	15
1.15 Memory Speed: DDR SDRAMS Memory Performance Ratings	15
1.16 What Type of Memory Module Does your Computer Use?	17
1.16.1 Crucial.com	17
1.16.2 Visual Inspection	17
11.16.3 nstalling DIMMs	17
2.0  Virtual Memory Operating Systems, Virtual Addresses and Paging	18
2.1 Introduction to Virtual Memory	18
2.2 Dynamic Address Translation	20
2.3 Format of a Virtual Address	21
2.4 pages and slots	21
2.5 Virtual Memory Paging and Thrashing	22
3.0  Introduction to Memory Management	23
3.1 Address space, ASID, jobs, tasks, processes, etc.	24
3.2 Address space and Addressability	24
3.3 Line, Bar and 64-bit Address space	25
3.4 Memory Isolation	25
3.5 Storage Protect Keys	26
4.0 Introduction to Task Management	27
4.1 Multitasking and Time Sharing	27
4.2 Subsystem, Daemon, and Service	28
4.3 SDSF DA command	28
4.4 z/OS Master Scheduler, Windows BOOTMGR and Linux init	28
4.5 Windows Process IDs	29
4.6 Linux Process IDs	30
4.7 Z/OS Jobs versus Processes	31



1.0 System Memory Architecture  and Memory Failures
Computer Memory Concepts - http://www.computermemoryupgrade.net/index1.html
Maximum Memory, Standard Memory and Slots -  http://www.computermemoryupgrade.net/maximum-standard-memory-slots.html
Memory RAM Speed - Access Time, Megahertz (MHz), Bytes Per Second -  http://www.computermemoryupgrade.net/measuring-ram-speed.html
System DRAM Technologies -  http://www.computermemoryupgrade.net/system-dram-technologies.html
DIMM - http://en.wikipedia.org/wiki/DIMM
Double Data Rate - http://en.wikipedia.org/wiki/Double_data_rate
DDR - http://en.wikipedia.org/wiki/DDR_SDRAM
DDR2 - http://en.wikipedia.org/wiki/DDR2_SDRAM
DDR3 - http://en.wikipedia.org/wiki/DDR3_SDRAM
DDR4 Intel set to bring next-gen DDR4 DRAM memory to computers later this year -  http://www.pcworld.com/article/2085880/intel-set-to-bring-ddr4-dram-to-computers-in-third-quarter.html
Intel plans to bring speedy DDR4 DRAM to computers in Q3 -   http://www.computerworld.com/s/article/9245266/Intel_plans_to_bring_speedy_DDR4_DRAM_to_computers_in_Q3
Everything you want to know about Dual- Triple and Quad Channel Memory Architectures - http://www.hardwaresecrets.com/article/133
Multi-channel memory architectures -  http://en.wikipedia.org/wiki/Multi-channel_memory_architecture
Dual Chanel Architecture - http://en.wikipedia.org/wiki/Dual-channel_architecture

Video - PC Memory Tutorial - http://www.youtube.com/watch?v=wZV7UV31VGg
Video - Understanding and upgrading your RAM - http://www.youtube.com/watch?v=e68YyNTeL18
Video - Installing your PC's memory - http://www.youtube.com/watch?v=Hw-zreo1hBE
Video - Learning about Memory (RAM) - HP Workbench - http://www.youtube.com/watch?v=WJ5kRofuGLQ
Video - How Much Memory (RAM) do I Need? - HP Workbench - http://www.youtube.com/watch?v=QNrzycPDwvk
Video - Guide to RAM Memory Channels as Fast As Possible - http://www.youtube.com/watch?v=-D8fhsXqq4o
Video - PCI Express (PCIe) 3.0 - Everything you need to Know As Fast As Possible - http://www.youtube.com/watch?v=LSSHuMHbCWo
Video - Dual & Triple Channel Memory - http://www.youtube.com/watch?v=ARi52co3ANs




1.1 Mom, I Learning How to Program
Before continuing, let us re-enforce the concept of computer programs and software. The concept computer programming should more appropriately be described as computer "teaching". This implies many things

First, before any teacher can teach they must have knowledge of the subject that they are teaching. The computer programmer must have the same subject knowledge as Bob.

Second, computers are "dumber" than any human student. Many of us assume that computers have infinite knowledge, but in reality they are really dumb.

Third, most students not only understand English they can tolerate and overcome small teacher errors. If your instructor misspells a word or forgets a period at the end of a sentence, you could probably adjust for the error and still understand the lesson.

Computer program languages have very precise syntax rules. If you earned a 99% on a History exam, you would be considered an A+ student. If you earned a 99% on a Chemistry exam, you would be considered an A+ student. If you write a program that is 99% correct, the program may not work at all or may even produce incorrect results. Like medical doctors, computer programmers must learn to be 100% correct while trying to teach dumb computers what to do.

If a programmer misses something, e.g., zero account balance, unexpected consequences may occur - as the following ConEdison story illustrates.

Back in the early 70s Con Edison, the giant public electric utility company servicing New York City, introduced the first public utility computerized electronic billing system.

George went aboard for business travel for several months had his electric system turned off. On his return he encountered a stack of electric utility bills and a First Disconnect Notice stating that he must pay his balance of $0.00 Dollars in Full.

Several weeks later, George received a Second Disconnect Notice. Again the notice stated that he must remit for overdue balance of $0.00 Dollars. George called up ConEdison and complained.

Finally, George received a Last Disconnect Notice stating that unless he remitted the overdue balance of $0.00 Dollars, his electric service would be terminated. George immediately wrote a check for $0.00 Dollars and sent in the overdue balance of zero.

You would glad to know that on Georges next month electric bill it stated the following, "Thank you for remitting your payment of $0.00 Dollars".

The demand for qualified computer programmers is at an all-time high. Students in entry level programming become very frustrated because 99% accuracy has meant "success" to them. When a student's program does not work, they blame themselves or better yet the teacher. But, they are not the problem. Student programmers must learn to keep their sense of humor.

It takes a student programmer two years to learn a programming language well enough for an entry level career opportunity at $55,000 and up. It takes an experienced programmer three-to-six months to learn their second computer programming language. Five-year experienced programmers are making 75-90K and up.

While learning computer programming does have its initial frustrations, there are very few other careers that has a starting salary level, job retention and promotion possibilities with just two years of initial training.



1.2 Overview of RAM Technologies

The type of RAM to be used by your computer is limited by your motherboard (system board). While a motherboard is designed to mount a particular type of processor, but mother board, and its chipsets determines the type and amount of RAM that can be installed with your computer. The following summarizes the more important issues:

	Form Factor: Type of memory module supported, e.g., SIMMs, DIMMS, RIMMS, SDI DIMMS, DDR DIMMS, and DDR2 DIMMS
	Type of memory technology supported, e.g., SDR, DDR, DDR2 SDRAMS
	Speed of memory supported as measured in nanoseconds, GHz, or PC rating.
	Type of error checking, e.g., EEC, parity or none.
	Memory slot population rules, e.g., maximum memory, Dual Channel Support, etc.
	CL or RL rating limitations

Form Factor Memory Modules and Memory Slots
Memory modules are installed into one or more motherboard memory slots. Either your motherboard installation book or your computer manual will specify the number and types of memory modules you can use. More experienced users will simply open the computer cabinet and look at the motherboard. The web site www.crucial.com can also help you determine the type of memory module used for your computer and motherboard.

The type of memory module supported can be visually identified by the number of pins and the location of a mounting notch. It is impossible to mount an incorrect memory module into your motherboard.











1.3 Memory Arctitecures:  Static Versus Dynamic

In addition to form factor, the chip technology used on the memory module defines the type of RAM.

1.3.1 SRAM (static RAM)
	This type of memory maintains data in storage as long as it is powered.
	Because it is faster and more reliable and expensive than DRAM, SRAM is most often used as cache memory.
	Earlier forms of SRAM were classified as Async or Sync RAM if they were synchronized with the system clock.
	The most common type of SRAM today is pipeline burst SRAM, which can operate at higher bus speeds.

1.3.2 DRAM (dynamic RAM)
	This type of RAM retains its data by being continuously rewritten (refreshed) every few milliseconds. DRAM transistors are charged by capacitors that must be re-charged
	DRAM is the most common type of expandable memory in workstations and servers today, but there are several different types of DRAM.

1.3.3 SDRAM (dynamic RAM)  and DDR

Synchronous dynamic random-access memory (SDRAM) is any dynamic random-access memory (DRAM) where the operation of its external pin interface is coordinated by an externally supplied clock signal.

DDR (Double Data Rate) SDRAM operates twice the system clock speed.


1.4 SIMM versus DIMM Form Factors

There is a wide variety of DIMM memory modules (168, 184 or 240 pins) and memory architectures (SDRAM, DDR, and DDR2). Before addressing these issues, one should first understand the simple difference between SIMMs and DIMMs.

	SIMMs (Single Inline Memory Modules) could only mount memory chips on one side of the module.

	DIMMS (Dual Inline memory Modules) could mount memory chips on both sides of the module by using two sets of independent pins. This change significantly increased the amount of memory that could be mounted on one module.

Probably the most confusing concept to distinguish the difference between memory modules and memory technologies.

A DIMM is a type of module that can mount memory chips on both sides. On the other hand, SDRAM (Synchronous Dynamic RAM) is a type of memory technology (type of chip) that is mounted on DIMM module.

1.5 Dual Channeling

DDR and DDR2 DIMM performance can be increased by using a motherboard that can use a feature called dual channeling. As you can see below, a memory controller can be access at the same time, thereby doubling the effective access speed and data width.

For example, instead of accessing data in 64-bit chucks it accesses data in 128 bit chucks (two 64-bit busses). Dual channeling does not increase the speed of the transfer, it just makes the highway (the bus) wider.




	Currently only DDR RAM technologis can take advanatged of  dual-channeling.
	For each channel, two DIMM modules that match in size, speed and features must be used, else the dual-channeling feature is disabled. Matching the manufacturer is also recommended.
	A motherboard that uses dual-channeling will clearly distinguish which slots are Channel A and Channel B (generally by color).
	Onee pair of memory modules inserted into Channel A does not have to match pair of memory modules inserted into Channel B.

1.6 Dual Channel Versus Double-Sided Memory

As stated previously, a DIMM module can have memory chips installed on either one side (one-sided) or both sides (double-sided). Double-sided DIMMS share the same 64-bit path and hence does not increase memory access.

Double-sided memory increases the memory size, but not access speed. Dual Channel memory increases access speed, but not memory size.

1.7 Error Checking: Parity

One special feature available in various memory technoplogies is the ability to detect memory failures and in some cases correct this failure.

The internal structure of a memory chip is roughly similar to a light switch. It is either off or on. When your home light switch does not work, it is easy to detect the problem - Hey, were did the lights go? But, when a memory module light switch, e.g., the DRAM transistor fails, how do we know that there is a problem?

Error checking is a hardware feature of a memory module that detect memory failures.

Older memory modules used an error checking technology called parity. Newer memory modules use a technology called ECC (error-correcting code).

1.8 Error Checking: ECC

Earlier methods of error checking called parity did not detect all types of memory failures. And when a error was detected, the system halted and if you had not saved your work, your work was lost.Current ECC methods can better detect memory errors and are called "self healing". Seal healing means that ECC memory can correct some types of errors automatically without halting your system. But, even newer ECC methods can not correct doubt-bit errors and data may be still loss.

ECC memory cost slightly more than non-ECC memory.

ECC memory most likely will have a odd number of chips installed on the memory module. DIMM ECC modules are identified as a 72-bit module instead of the 64 bits, e.g., a multiple of 9 instead of 8. RIMM modules identify ECC memory as x18 and non-ECC memory as X16.

Your motherboard must support ECC memory and in many cases the ECC memory feature must be manually activated in the CMOS setup.

1.9 Limitations of Error Checking and Error Checking Diagnostic Tools

Does memory error checking guarantee that I do not loose my work (data)? Yes. So I Loose My Work Anyway even though the falure was detected? Yes.

If you do not frequently save your work or properly set auto-save features correctly, error correction will not stop you from loosing data.

So what good is ECC memory if the system stops without saving my work? The main advantage is that the system did stop. It did stop before corrupting existing data stored on your disk. One of the most damaging result of memory failure is data corruption. Hence, ECC memory is frequently installed in servers are other mission-critical applications.

Most memory checking tools, to be presetned later, will conduct a "stress test" to determine if there is a memory failure has occured.  Newer Predictive Failure Analysis (PFA) Tools collects data and uses aritfical intellegince routines to predict failures before they damaage your system.  PFA tools are comparable to visiting your doctor each year to conduct diagnostic screening to prevent future health problems.

1.10 IBM z System Predictive failure analysis and S.M.A.R.T.

Predictive Failure Analysis (PFA) refers to computer mechanisms that analyse trends in corrected errors to predict future failures of hardware components and proactively enabling mechanisms to avoid them. Predictive Failure Analysis was originally used as term for a proprietary IBM technology for monitoring the likelihood of hard disk drives to fail, although the term is now used generically for a variety of technologies for judging the imminent failure of CPU's, memory and I/O devices. A

S.M.A.R.T. drive monitors the internal performance of the motors, media, heads, and electronics of the drive, while our software monitors the overall reliability status of the drive. S.M.A.R.T. is an industry standard for monitoring and reporting fault conditions in a peripheral storage device, whether from damage or normal wear-and-tear. S.M.A.R.T. is intended to decrease the chance of data loss by giving adequate warning to these conditions, but it is no replacement for good disaster-recovery planning. A S.M.A.R.T. drive monitors the internal performance of the motors, media, heads, and electronics of the drive, while our software monitors the overall reliability status of the drive.


IBM z System is the leader in PFA technologies applied to processors and RAM by a significant margin. Intel, AMD, HP and others have recenetly intrdouced these technologies and will continue to I mprove.  High counts of corrected RAM intermittent errors by ECC can be predictive of future DIMM failures and so automatic offlining for memory and CPU caches can be used to avoid future errors, for example under the Linux operating system the mcelog daemon will automatically remove from usage memory pages showing excessive corrections, and will remove from usage processor cores showing excessive cache correctable memory errors

1.11 Symptoms of Memory Failures

1.	When you first turn on your computer it runs fine, but as you go about your business you notice that its performance diminishes. By lunch time, websites take minutes to load and local programs run at a snail's pace. This type of gradual deterioration of PC performance, especially with memory-intensive programs, may be caused by a RAM problem.

2.	Since almost all of the data running on a computer passes through the system memory, data corruption is one indicator of a failing memory module. Common symptoms include lost, missing or scrambled data; frequent crashes; spontaneous reboots; registry errors; missing system files; and other intermittent or bizarre behavior.

3.	Your computer randomly restarts while you are in the middle of something or freeze sporadically. It may also reboot almost immediately upon opening the desktop. This could be a sign of faulty RAM.

4.	A blue screen with white text flashes before restarting. Blue-screen errors are annoying because you don't even have a chance to read the error message. Bad RAM is one thing that cause them. "blue screen" system crashes may  reference memory read/write, parity, interrupt, address, page faults, general protection faults or mismatch errors. In some cases, the system may simply refuse to recognize the installed memory.

5.	Data corruption.  Files—particularly ones you frequently access and save—seem to be inexplicably corrupted. RAM issues can lead to this problem, which can worsen over time. The file structure of your hard drive may slowly degenerate and you will no longer be able to boot your machine.

6.	Your attempts to install a new program repeatedly fail for unknown reasons. You try to reinstall the operating system, but keep getting odd error messages.

7.	Boot failures. Because the BIOS checks the computer memory every time the system boots, memory failures often result in an unbootable system that may present with a blank screen and a system beep.


1.12 What Causes Memory Damage?

1.	Power surges can damage most computer components, including RAM. You should plug your computer and other expensive electronics into a surge protector. Make sure you know the difference between a surge protector and a power strip.

2.	Before you handle any parts in your computer, make sure you ground yourself by touching a piece of grounded metal to discharge static electricity. Electrostatic discharge can damage your computer.

3.	Excessive heat can cause RAM and other parts to wear out over time. Individual components can overheat, or heat from one component can cause damage to adjacent parts.
4.	If you have overclocked any part of your computer incorrectly, it may cause damage in the form of excess heat.

5.	Your memory module may have some fault that passed through quality control and worsened over time. This is the most likely cause behind a damaged RAM.

6.	It is also possible that the memory module is fine, but one or more memory slots on your motherboard are defective, hindering the RAM's performance. The defect may even be so bad that it damages the memory stick.

1.13 How to Test Your Windows Computer’s RAM for Problems

1.13.1 Windows Memory Diagnostic Tool

Is your computer unstable? There may be a problem with its RAM. To check, you can either use a hidden system tool included with Windows or download and boot a more advanced tool. To launch the Windows Memory Diagnostic tool, open the Start menu, type “Windows Memory Diagnostic”, and press Enter.





1.13.2 CPU-Z  Tool

If you’re looking for a more powerful testing tool, you can download and use MemTest86. It performs a wider variety of tests and may find issues that the included Windows test won’t. The latest releases of this tool offer a paid version with more feature, although the free version should do everything you need. You don’t need to pay for anything. MemTest86 is signed by Microsoft, so it will work even on systems with Secure Boot enabled.

So, start installing CPU-Z. Once done, you now need to start CPU-Z…




1.14 Memory Speeds
	Effective memory speeds are effected and rated by several factors.
	Nanosecond Ratings - internal speed to change one bit
	MHz Ratings - Clock speed that is synchronized with the processor
	PC Ratings - Amount of data as measured in MPBS that is transfer between the processor and memory; combines MHz and the width of the bus as measured in bits, e.g., 16, 32, and 64-bits.
	CL and RL Rating - internal speed to change a column or row of bits
	Dual Channel - ability of the memory controller to access two memory modules at the same time. Not all modules support dual-channel

1.4.1 Memory Speed: Nanosecond Ratings

A Nanosecond rating measures is the amount of time to change a bit status from off to on (memory refresh time). This is the internal speed of the chip.  Nanosecond ratings were important for older technologies, but most modern SDRAMs operate at 8.5 nanoseconds.  Nanosecond ratings have been replaced with CAS and RAS Latency ratings.

Memory Speed: MHz Ratings

	Memory speeds of SDRAMs, DDRs, DDR2s, and RAMBUS are measured in MHz. This MHz memory rating not the speed of your processor, but rather the speed of the Front Side Bus. The Front Side Bus is the connection, or road way, between your processor and the memory controller and RAM. The faster the FSB, the faster RAM can be accessed.
	One million hertz represents one million clock cycles per second. Older technologies can transmit one bit per cycle. Newer technologies can access two or more bits be cycle.
	MHz ratings do not adjust for the differences in the FSB width or dual-channeling, e.g., 16-bit, 32-bit, 64-bit and 128-bit (dual channel).
	The Front Side Bus is determined by the architecture used by you motherboard
	RAM MHz rating are frequently displayed as DDR266 which indicates a 266 MHz.

1.14.2 Memory Speed: PC and PC2 Ratings

	A PC Rating is the measure of the total bandwidth of data movement between the processor and RAM as measured in MBPS or GBPS.
	Unlike MHz, PC ratings take in consideration of the width of the FSB, and hence is a more accurate realistic measurement of Memory Speed.
	A 64-bit DIMM module can transmit 8 bytes at one time over the FSB. If the Clock Speed is 266 MHz, then the PC rating would be 2128 MBPS, or PC 2100.
	DDR RAM frequently is rated as PC XXXX, while DDR2 RAM may be rated as PC2 XXXX.

1.14.3 Memory Speed: CL and RL Ratings

	CAS Latency (CL) - Column Access Strobe is the number of cycles required to read a column of data in a memory chip.  CL is the more popular measurement then RL
	RAS Latency (RL) - Row Access Strobe is the number of cycles required to read row of data in a memory chip.
	The lower the CL or RL rating the better.  Latency values are represented by a series of numbers, e.g., 5-5-5-15. The first number five is the CL rating. The second number 5 will be the RL rating.

1.15 Memory Speed: DDR SDRAMS Memory Performance Ratings

DDR, DDR2, and DDR3 are rated using by two specifications PC or PC2 and DDR or DDR2.

DDR-xxx and DDR2-xxx specifications indicates the chip data rate, which is based on the clock speed of the Front-Side Bus. Using the PC2-3200/DDR2-400B DIMM in the DDR2  Performance Ratings below let us assume the Front-Side Bus speed is 200-Mhtrz. The FSB clock speed is determined by the technology used by the mother board. The FSB speed is always significantly slower that the processor speed.

At first glance, one may assume that at 200-Mhrtz we can get 200 million units of work done.  But here is the beauty of the DDR (DOUBLE DATE RATE) technology – is double-pumped or double-rated.    When the FSB bus operates with double data rate transfers data it performs twice the work of the clock speed, e.g., once of the rising voltage of the pulse and once on the falling voltage of the clock signal. It kinda like your boss making you work twice as hard during your 8-hour shift.

Double-channel technology is NOT double-rated technology. Normally the processor has only one path from the processor to RAM (one FSB). Double-channel means that there are two FSB busses.  With double-channel technology two transfers can be made simultaneously. Again, it may be like your boss giving you an extra worker to help you during your 8-hjour shift.

The bottom line is a 200-Mhtrz front-side bus gets 400-Mhrtz of work done. Again the speed of the FSB is determined by your motherboard.

PC ratings measure the maximum data transfer rate (not the chip data rate).  PC ratings are more likely be used to purchase new RAM.  A PC2-3200 indicates that the maximum data transfer rate of the DIMM module is 3,200,000 bytes, not bits. Some of the PC ratings truncate the last two digits. For example, the PC2-4200 DIMM modules actually has a maximum data transfer rate of 4,266,000 bytes.

DDR and DDR2 PC ratings, the Chip Data Rate and the Front –Side Bus speed are directly related. Consider the following:

PC Rating   divided by 8   equals   Chip Data Transfer Rate.   divided by 2  equals   Front-Side Bus Speed
PC2-3200            /8                                       400                                 /2                                             200 Mhrtz

Ok, Ok. Why do we divide by 8?  The Front side but is 64-bits wide. A byte is 8 bits.   64 /8bits = 8 bytes
Ok, Ok. Why do we divide by 2?  Did you forget double pumping or the meaning of double rated chip already?

We can also perform the conversion the other way.

Front-side bus speed   multiply by 2   equals  Chip Data Transfer Rate     multiply by 8  equals  PC rating.
        200 Mhrtz                       x2                                             400                                 8                       PC2-3200

DDR2  Performance Ratings

DIMM Module Name	DIMM Maximum  Data Transfer Rate	DDR Chip name	Chip Data Transfer Rate	I/O Clock

PC2-3200	3200	DDR2-400B
DDR2-400C	400	200
PC2-4200*	4266	DDR2-533B
DDR2-533C	533	266
PC2-5300*	5333	DDR2-667C
DDR2-667D	667	333
PC2-6400	6400	DDR2-800C
DDR2-800D
DDR2-800E	800	400
PC2-8500*	8533	DDR2-1066E
DDR2-1066F	1066	533



1.16 What Type of Memory Module Does your Computer Use?

1.16.1 Crucial.com

You can visit the crucical.com web site to detemine the type and the total amount of memory you computer's mother board  supports. Most lower-priceddesktops and laptops will include 8 gigabytes of
Memory, but the memory may be expanded by either adding memory to unused memory slots (which is rarely the case), or by replacing existing memory modules with higher capacitly modules.

1.16.2 Visual Inspection




11.16.3 nstalling DIMMs






2.0  Virtual Memory Operating Systems, Virtual Addresses and Paging

Video - z/OS System Memory Architecture - Unit-10 https://youtu.be/viLXEL27R2s

IBM Paul Newton's presentation and demonstration of z/OS address spaces, virtual storage and paging, and using key-controlled protection for system integrity.

See Blackboard Folder "Paul Newton's z System Presentations" for a copy of this Presentation


Virtual address space - http://en.wikipedia.org/wiki/Virtual_address_space
What is virtual memory, how is it implemented, and why do operating systems use it?
http://www.programmerinterview.com/index.php/operating-systems/how-virtual-memory-works/
What are the advantages and disadvantages of virtual memory?
http://answers.yahoo.com/question/index?qid=20080928221645AAFAg23


These videos have been written for Unix, but many of the concepts can be applied to z/OS
Video - Virtual Memory - Part 1 - http://www.youtube.com/watch?v=iqfjWIYyQ9k
Video - Virtual Memory - Part 2 -  http://www.youtube.com/watch?v=4px3RrDBa6w
Video - Page Replacement -  http://www.youtube.com/watch?v=M0d-vLw5Eyk
Video - Process Management -  http://www.youtube.com/watch?v=mHPySA51t18
Video - Processes - Part I -  http://www.youtube.com/watch?v=TIa2mhKCeYo
Video - Processes - Part II -  http://www.youtube.com/watch?v=_5EV7isUJ6k

2.1 Introduction to Virtual Memory

All modern operating systems, like z/OS, Windows and Linux uses virtual addresses to locate programs and data in z/OS address space. A virtual address is divided into three areas: Region, Segment and Page. z/OS uses these concepts to in dynamic address translation find the virtual location of a specified area of memory. While at first these concepts may sound confusing assume that visitor from Ireland wanted to find my home. Would they start searching at the first street in New York City - obviously no

First, we would start by asking which state do you want to locate (which Region)? Second, one would ask which city within the state do you want to locate (the segment table)? Third, which street within the city do you want to locate (the page table)? Finally, one can sequentially search all houses on the street or simply use the house address (the byte offset).  Likewise, a virtual address uses a similar approach to unique identify a virtual address location and to quickly find the address later when needed.

Many modern laptops and servers are based on a 64-bit architecture. Compared to a 32-bit processor a 64-bit processor can process more data with each computer cycle, has a larger and more powerful instruction set, and can “theoretically” access 16 extabytes (EB) of physical or virtual memory. But, most laptops have only 8 or 16 gigabytes (GB) of physical memory.
While a 64-bit processor can access 16 extabytes of memory, no server can have 16 extabytes of "installed" physical memory.  IBM introduced a virtual memory management operating system (OS/360) in mid-1960s to enable a system to execute programs larger than the installed physical memory. Decades later most other computer hardware and operating systems also introduced Virtual Memory Management.
Technical Review – Virtual Memory Management (VMM) is a technique that enables a computer system to execute programs and data larger than the installed physical memory.  During program execution unused pages (sections) will be temporally stored on disk or in flash memory storage and automatically returned to physical memory as needed during execution. The advantage of VMM is to be able to execute larger programs or a greater number smaller programs with limited physical memory.
Using the analogy of a student preparing a major research paper on their classroom desktop one may better understand how Virtual Memory Management works. During the process of preparing the research paper, the student has mountains of borrowed library research books, document copies, and other reference material so physically large that not all of these reference materials can fit on the student’s desktop. As a result the student may temporarily place some of the overflow research reference materials on a separate table, book bag, or even the floor to be read later. The student does not want to return the borrowed research books to the library because they are in the "process" of working on the research paper.
An application program similarly needs to be stored in physical memory, like a desktop, to be executed. When the physical memory is full, a virtual memory operating system will temporarily store pieces of the program called "pages" on a disk or in flash memory. Each executing program page will be assigned a unique identifier called a virtual memory address. During program execution an operating system table will remember the current location of the page. Is the page, or virtual memory address, stored in physical memory? Or is the page, or virtual memory address, stored on the disk?
When the page is required to be executed, the virtual memory operating system automatically returns the application program page to be executed. Virtual memory operating system can execute larger or a greater number of programs that exceed the capacity of physical memory. The bad news is that using the disk for the virtual memory overflow slows down the net processing performed. https://en.wikipedia.org/wiki/Virtual_memory

Applying a library analogy, is the library book on the book shelf or is the book "out on loan" being read by another student? The library book has an identifier similar to a virtual memory address. The librarian maintains a book inventory which lists the current status and location of the book similar to the use of a virtual memory list which specifies the status and location of a program page in physical memory or on disk.



2.2 Dynamic Address Translation




2.3 Format of a Virtual Address




2.4 pages and slots.

Assume that the following program has be divided into eight pages labeled A, B, C, D, E, F, G, and H. The Virtual Memory table contains the address of ALL program pages.  Currently, pages A, E, F, and H are located in real storage. If Page A requests, e.g., GO TO, page H, the virtual memory manager needs to consulted to determine the location of Page H. The virtual memory manager determines the ASID of the page making the request. (Remember, the ASID identifies the address space contains all of the real and auxiliary pages  Each address space is assigned a  Region, Segment and Page table  maintained by the virtual memory manager.




Since the virtual memory manager determines that Page H is in real memory it tells the physical memory address the real address of the FRAME. If Page B was needed the virtual memory manager would assign the disk address of the SLOT to the auxiliary memory manager.

Frames and slots are constantly swapped between real and auxiliary disk storage. Swapping and page stealing are various swapping algorithms.  The objective is to minimize the overhead process of moving pages between physical and auxiliary storage. The system programmer can designate that some programs may not be swapped, Since the virtual, physical and auxiliary storage managers are so critical to the operation of z/OS these processes may not be swapped,


2.5 Virtual Memory Paging and Thrashing

When the operating system needs more physical room space it copies unused pages of program code and data to your hard disk.
When a program needs a page that is not in main memory, the operating system copies the required page into memory and copies another page back to the disk. One says that the operating system pages the data. Each time a page is needed that is not currently in memory, a page fault occurs. An invalid page fault occurs when the address of the page being requested is invalid.
Virtual memory is slower than RAM because disk access is by itself slower than RAM. However, another problem call "thrashing" may occur when the operating system does more paging than it does running your programs. Adding more RAM to your computer may increase the performance of your computer more than upgrading a processor.
Before you add more RAM you can alter the Windows Virtual Memory settings to increase the amount of virtual memory or to move the virtual memory swap file.



3.0  Introduction to Memory Management

Every operating system it responsible to manage memory in which programs are stored and execute system and user tasks (programs). View the internal memory of a computer as collection of electronic storage boxes. Just like your personal mailbox address uniquely identifies your home and is assigned to you, a computer's address must uniquely identify a storage area and assign to a task or a job.

Each physical computer has a physical limit to amount of real memory and all operating systems  and application programs must in compiled in a way to respect that limit. The size of your personal mailbox has a physical limit to amount of mail you may store in it. Computer systems identify the physical or virtual limits in based on the processor architecture. A 16-bit computer system has a limit of one megabyte, the 32-bit architecture limit is 3.4 gigabytes, and 64-bit architecture limit is 16 exabytes.

While many modern operating systems, e.g., Windows, Linux, UNIX, etc. have been designed to support 16-bit, 32-bit and 64-bit limits, these operating systems often are not backward compatible.  In simpler terms, there are often some limitations to execute older programs due to memory addressing, interface or security issues. On the other, the 64-bit z System is designed to maintained and execute all applications no matter which type of architecture the programs were designed.

Over the years, the mainframe was designed to support 24-bit addressing (below the line addressing), 31-bit addressing (below the bar addressing), and 64-bit addressing (default addressing).  The application program and JCL parameters RMODE and AMODE are used to select which addressing scheme should be emulated on the 64-bit mainframe.

Application program backward compatibility is important to large scale organizations. Windows 8 had many serious backward compatibility limitations, in which older applications could not execute. This meant that either original application had to be written or a completely new compatible application needs to be purchased, users trained, and operational procedures redesigned.

This is not acceptable in enterprise systems.  Many applications have been in service for thirty or forty years. Suites of these applications were designed, coded and tested at the cost of billions of dollars and years of development time. For example, many commercial banks still use business logic COBOL applications developed in the 70s and 80s. The business logic has not changed, only the user interface has changed. With the assistance of web services, or messaging, a mobile app can communicate with a time-proven COBOL banking application.

How Operating Systems Work - http://computer.howstuffworks.com/operating-system.htm
What is an address space?
http://www-01.ibm.com/support/knowledgecenter/zosbasics/com.ibm.zos.zconcepts/zconcepts_82.htm
System address spaces and the master scheduler - http://www-01.ibm.com/support/knowledgecenter/zosbasics/com.ibm.zos.zconcepts/zconcepts_105.htm
Address space - http://searchstorage.techtarget.com/definition/address-space
Process identifier - http://en.wikipedia.org/wiki/Process_identifier



3.1 Address space, ASID, jobs, tasks, processes, etc.

Most modern operating systems are multitasking. This means that the system memory must be subdivided to accommodate different tasks programs stored in memory.  An address space is the complete range of addresses (computer memory) available to a program to store executable code and data.  For example, one address space may be assigned Microsoft Word, which contains the Word application program and your document (program data). A second address may be assigned to Internet Explorer application program and the display web page (program data).

It is important for an operating system to uniquely identify each executing program and the address space assigned to that executing program. An Address Space Identifier (ASID) is a unique number assigned to an address space which is used to track the storage location of a program. Since an individual program may be executed simultaneously, each copy of the program must be assigned separate address space.  z/OS uses the term JOB to describe an individual  program that was start executing sequence of processes or tasks. Therefore, the concept of ASID is related to memory management (where is the executing program stored in memory) and JOBID  is used by task management to execute each step of the job.

Windows, UNIX and Windows use an alternative unique descriptor, Process Identifier (PID), to uniquely describe an executing program which will be stored in some address space. The Windows and Linux PID is conceptually similar to the z/OS Job ID

3.2 Address space and Addressability

Addressability represents a program's ability to access or reference all of the storage associated with an address space, e.g., programs and data.

Another way of thinking of an address space is as a programmer’s map of the virtual storage available for code and data. An address space provides each programmer with access to all of the addresses available through the computer architecture, defined this characteristic as addressability). z/OS provides each user with a unique address space and maintains the distinction between the programs and data belonging to each address space. Because it maps all of the available addresses, however, an address space includes system code and data and user code and data.  Thus, not all of the mapped addresses are available for user code and data.

3.3 Line, Bar and 64-bit Address space




The original operating space use a 24-bit addressability (Below the Line) permitted a program to access 16MB of memory.  31-bit addressability (Below the Line) increases a program's ability to access 2GB of real or virtual memory. 64-bit addressability, introduced in 2000, increases a program's ability to access 16TB of real or virtual memory for z/OS, Unix System Services and zLinux.

64-bit addressability was first introduced on super computers in 1985, such as the Cray family of computers. 64-bit addressability was introduced by Solaris UNIX in 1998, Linux and Mac OS in 2003, and Microsoft in 2005. A 64-bit architecture is dependent on several concepts: 1) processor design, 2) operating system addressability, 3) application addressability, and 4) backward capability. The history of 62-bit architectures includes more products in which 64-bit addressability was more "smoke and mirrors" than substance. However, the evolution of 64-bit addressability and virtual memory has improved into a stable and powerful architecture.

However, z System addressability is the only architecture that is fully backward compatible for ALL 24-bit, 31-bit and 64-bit applications codes since 1952. For example, even Windows 8 and Windows continues to be unable to execute many older 16-bit and 32-bit programs.  When stat a JOB, you can specify which addressing mode by specifying the JCL parameter AMODE to indicate which hardware addressing mode, i.e., 24-bit, 31-bit or 64-bit to be used when the program executes.

3.4 Memory Isolation

It is very important, the address space of the Word application program and data be separated and isolated from the address of Excel application program and data, just like you do not want virus not to invade the address space of an executing application or stored data on your disk. However, many operating systems were written in the C programming language, and these operating systems were unable to directly protect and isolate address space. While Windows, UNIX and Linux have introduced many solutions to improve address space isolation, no other solution is comparable close to the level protection and performance of z System Storage Protect Keys, which provides a hardware and software memory protection system for over 40 years.  Since Windows, UNIX and Linux are designed to support any hardware platform, they cannot provide the same hardware enforced memory protection as z/OS Storage Protect Keys.  Windows Vista introduced a secured kernel in 2003 and Linux introduced SeLINUX (Security-Enhanced Linux) in 1998, but neither operating system provides hardware protection. IBM software and hardware Storage Protect Keys have been in use for over 50 years.

Many programs and users are competing for the use of the system. So how does z/OS® preserves the integrity of each user's work? One technique is through the use of multiple storage protect keys.

3.5 Storage Protect Keys

Under z/OS, the information in central storage is protected from unauthorized use by means of multiple storage protect keys. A control field in storage called a key is associated with each 4K frame of central storage.

When a request is made to modify the contents of a central storage location, the key associated with the request is compared to the storage protect key. If the keys match or the program is executing in key 0, the request is satisfied. If the key associated with the request does not match the storage key, the system rejects the request and issues a program exception interruption.

When a request is made to read (or fetch) the contents of a central storage location, the request is automatically satisfied unless the fetch protect bit is on, indicating that the frame is fetch-protected. When a request is made to access the contents of a fetch-protected central storage location, the key in storage is compared to the key associated with the request. If the keys match, or the requestor is in key 0, the request is satisfied. If the keys do not match, and the requestor is not in key 0, the system rejects the request and issues a program exception interruption.

z/OS uses 16 storage protect keys. A specific key is assigned according to the type of work being performed. The key is stored in bits 8 through 11 of the program status word (PSW). A PSW is assigned to each job in the system.

Storage protect keys 0 through 7 are used by the z/OS base control program (BCP) and various subsystems and middleware products. Storage protect key 0 is the master key. Its use is restricted to those parts of the BCP that require almost unlimited store and fetch capabilities. In almost any situation, a storage protect key of 0 associated with a request to access or modify the contents of a central storage location means that the request will be satisfied.

By default ALL the z/OS operating system, subsystems and application program space are isolated. The Storage Protect Keys provide a mechanism that provides an application program the limited ability to access the address space operating system, subsystem and other application programs and data. Storage Protect Keys are assigned on a program -by-program basis. For example, the Storage Protect Key 0, also called the Master Key, means that those base control programs (system kernel) can access any address space, but very few programs can access its address space. How does this base control program give permission for others to access its address space? It assigns one or more protection keys to a list buddy's list FaceBook friend list (I think that is what they call it).  This Storage Protect Key List may state that I will talk with all programs who have a Storage Protect Key of 2 and 4, but no other program.

O.K. Why can't I compile my own application program with a Storage Protect Key of 0?  Good question. But, only IBM and third party vendors may use Storage Protect Keys 0 thru 7.  An IBM customer cannot.  On the other hand, IBM customer may use Storage Protect Keys 0 through 15 to create their own customized address space storage protection plan.   But, most applications use NO Storage Protect Keys since they require the default to provide the most security for memory isolation.



4.0 Introduction to Task Management

4.1 Multitasking and Time Sharing

The design of a multitasking operating system is to store multiple programs waiting to be executed in various address spaces (memory management) and then allocating processor resources to execute to programs (task management). In a multiprogramming operating system a program, identified by an ASID, starts execution of a program, but switches to another program (ASID) when the first program requests an I/O task to be delegate to a, I/O channel or other I/O processor. A similar procedure is used by a time sharing operating system, except the operating system switch processor control by allocated time.  When a processor switches processing control from one process to another process this is called a context switch.  In modern computer architectures, programs may share multiple processors or processor cores.   The basic concept of context switch will remain the same.

Most of us have a cell phone that permits one to switch between two or more phone calls. In the middle of one conversation, we switch to a second conversation. Then when we subsequently return to the first conversation we ask, "Where were we?" Likewise, when the processor switches between multiple executing programs the operating system needs a data structure to remember where the program (conversation) left off  when it switch to the execution a different program so that we can resume where it left off .  Where were we?

The actual data that stores information on where the process was at the point of the switch is call context or state. Hence, the process is called context switching.  Where is that context or state data stored to resume execution?   It is stored in a special operating system area called a control blocks, or tasks or process control blocks.



4.2 Subsystem, Daemon, and Service

It is important to understand that an address space is an electronic box, but the operating system does not know the content of that electronic box.   So far we have use the terminology of programs and data.   Most operating systems use other more detailed descriptors, such as processes or tasks, to describe a program that is assigned an address space, ASID or PID.  z/OS often uses the term JOB to describe a program that may start a sequence of processes or tasks. Windows uses the term Application to describe a process which has a user interface as compared to Windows processes which are stored in and address space the background waiting to be executed when some event occurs.  Operating systems use different terms which label a process waiting in the background, or has no direct user interface, or waits for an external process signal to start processing.  (The following examples are normally the foundation of a client - server architecture (requestor and provider).

•	Linux uses the term Daemon
•	Windows uses the term Service
•	z/OS uses the term Subsystem
4.3 SDSF DA command

The following process information is displayed by the SDSF DA command. Since the displayed information is wider than 80 columns, one must press F11 to scroll to the right to display the ASID column. *MASTER* JOBNAME has an address identifier of 1; this is the first job/process to be processed when the system first started, or IPL. This is the Master Scheduler subsystem program and Jobname .



4.4 z/OS Master Scheduler, Windows BOOTMGR and Linux init
The master scheduler subsystem, for example, runs in the address space called *MASTER* which will be used to establish communication between z/OS and its own address spaces. When you start z/OS, master initialization routines initialize system services, such as the system log and communication task, and start the master scheduler address space. The master scheduler subsystem is used to establish communication between z/OS and its own address spaces. When you start z/OS, master initialization routines initialize operating system services, such as the system log and communication task, and start the master scheduler address space. The Master Scheduler loads other operating system base control programs (kernel) and subsystems processes.

All operating system have a program similar to the master scheduler. Linux uses the init program to load the kernel of the operating system. Windows use the NTLDR or BOOTMGR programs to load the operating system kernel, depending on the version of Windows.

Other Interesting information.  SDSF DA display the ASID in two different columns. The ASID column displays the ASID value in decimal, and the ASIDX column displays the ASID value in hexadecimal. (More on hexadecimal later.)


When you power up an Android device, you clearly see the Linux init program executing. One of the subsystems will display the startup flash animated image that will entertain you during the startup process. The Windows NTDLR and BOOTMGR also display a startup flash animated image. With a little skill you can change the animated image.


4.5 Windows Process IDs

You can configure the Windows Task Manager to display the Process IDs  A PID is similar to the z/OS ASID. By default the Windows Task Manager does not display the PIDs of Windows initialization or kernel programs





4.6 Linux Process IDs

You can use the Linux process, i.e., ps,  to display the Process IDs  A PID is similar to the z/OS ASID. Linux displays a PPID, Parent Process ID, to indicate which parent process started this subprocess. The Linux init (system initialization program) like *MASTER* in z/OS always have the value of 1.  They were the first process to be executed.


4.7 Z/OS Jobs versus Processes

The following process information is displayed by the SDSF DA command.  The *MASTER* JOBNAME is actually a JCL program that initializes other processes.  JCL programs are always assigned a Job ID. In this example the *MASTER* Job is assigned the JobID ZSTC06987. During execution the *MASTER* job will load PCAUTH, RASP, WLM, CONSOLE, SMF, and OMVS executable load modules. Notice that these load modules do not have a JobID. This means that these processes were started by a JCL program. But, these processes were assigned a ASID.

The Linux init program uses a series of text files, e.g., init.d and runlevel files, to specify the processes to start. z/OS uses JCL programs to start the operating system.  To repeat the *MASTER* JCL or Master Scheduler is the first JOB to be executed. Not all processes have a JobID, but all processes has a ASID



PCAUTH Cross-memory support process
RASP Real storage manager process (includes advanced address space facilities support)
XCFAS Cross system coupling facility process
WLM Workload management process (to be studied later in Assignment 2)
OMVS - z/OS UNIX Shell process (to be studied later in Assignment 2