VERSION 2.

0 3rd July 2000

Day-wise Schedule
Details of the Sessions
Day 1 Session 1 (Theory) Computing Environment -Key Concepts & terminology, Typical IBM MF site. IBM Operating Systems IBM families of OS - IBM Hardware - MVS Implementation Demo on ISPF Familiarizing with various options of TSO & ISPF Editor MVS Functions Data Management Overview - Job Management Overview MVS Tools Overview Assignments ISPF editor & TSO
MVS

Session 2 (Lab) Day 2 Session 3 (Theory)

Session 4 (Lab)

COMPUTING ENVIRONMENT KEY CONCEPTS & TERMINOLOGY

MVS

Command Processing
Command Issue Mode
How a user (programmer / end-user) interacts with the computer E.g. To edit a program, to execute a program On-line Mode - Using Terminal Batch Mode - Using Punched Cards or JCL

Command Execution Mode

Foreground - Terminal is locked while the command is being executed Background - Terminal is free while the command is being executed

MVS

Data Processing
How Business applications are executed On-line End User performs business functions Application programs work interactively with End User Execution is in foreground mode Database is immediately updated to reflect the changes Typically used for transaction processing, queries, master updates functions

MVS

Data Processing
Batch
Application programs are executed in background mode Periodic business functions are executed automatically As and when business functions are triggered by End User Operations department is responsible for monitoring the

execution
A command file is created to execute these functions One command file may consist of multiple programs / system

utilities
Typically used for bulk transaction processing, report printing ,

periodic processing (e.g. invoice generation, payroll calculation)

MVS

Time Sharing
Resource Sharing Multiple Users compete for computer resources at the same time At any given point in time only one user can have control of the resources What should be the basis of sharing?
1. 2. 3.

4.
5.

First come first served? Priority based? Who so ever can grab it - Law of Jungle? Equal - Democratically? Need based?

Usually combination of 2 and 4 is used i.e. all are equal but some are more equal!!!!!
MVS

Time Sharing
Time Slice Each user is given control of resources for a pre-defined period - time slice The control is passed on to next in queue user at the end of time slice (even if first users work is incomplete) If the user requires I/O before the time slice is over, the control is handed over to the next user (since CPU cannot do anything until I/O is complete)

MVS

Time Sharing
Priority
Each user / function is assigned a priority level The higher priority users are serviced first in a round robin

fashion
Only if the higher priority users are in wait state for I/O

completion the users in the lower priority are serviced Time Sharing typically refers to sharing of resources in an interactive processing mode

MVS

Multi-programming
Why Multi-programming? The program has CPU based and Non-CPU based instructions CPU is kept waiting during the non-CPU based instructions execution E.g. I/O operations (Disk, Terminal, Printer) This results in wastage of CPU time - a precious resource Multi-programming results in better CPU utilization

MVS

10

Multi-programming
How Does it Work? Multiple programs are kept ready for execution CPU can execute only one program at any given point in time If the currently executing program requires I/O, it is put in a wait state Another program is immediately taken for execution On completion of I/O the program again becomes ready for execution

This results in an illusion that multiple programs are being executed simultaneously, hence multiprogramming.
MVS

11

Multi-programming
Multi-programming Overheads
Program Queue Management Program Status Management

Context Switching during Changeover

Multiple programs must be in main memory Management of Common Resource Sharing (e.g. Printer)

It is critical to determine optimum level of Multiprogramming to maintain certain service level.

Relevance of Multi-programming
Multi-programming is applicable even for single user system Multi-programming is a must for multi-user system

MVS

12

Multi-processing
There are multiple CPUs (processors) in one machine These work together under single operating system Each CPU executes a separate program O/S assigns programs to each CPU Essentially CPU is treated as an allocable device!!!!!

MVS

13

Spooling
Why Spooling?
Multiple programs may need same printer at the same time May result in intermixing of output Exclusive access to a program will hold other programs Printer is much slower, results in longer wait state

How it is Implemented?
Output to printer is intercepted and written to a disk i.e.

spooled
On completion of program spooled output is queued for

Printing
This queue is processed by O/S print routine The O/S print routine is multi-programmed along with

application programs
MVS

14

Virtual Storage
Why Virtual Storage
Required to enable execution of programs with are larger than the main

memory size

What is Virtual Storage

Technique to simulate large amount of main storage Reality is main storage is much less E.g. Real main storage is 16MB but virtual storage is 2GB

How Virtual Storage is Implemented

Program executable code is generated assuming virtual storage size Only part of the program is loaded in main memory Address translation mechanism is used to map virtual address to actual

address Feasible because only the instruction currently being executed and the corresponding data need to be in the main storage
MVS

15

Virtual Storage
Advantages of Virtual Storage Main memory can be shared by multiple programs Enables effective use of the limited main storage Overheads of Virtual Storage Address mapping Keeping track of what is in memory and what is not Data/Instructions need to be brought in main memory as an when required Remove from main memory what is not currently required (to make room for instructions of other program) Memory Management
MVS

16

Computing Environment - Typical IBM MF Site

Business Environment

Large Local or Global Operations or Both User Community in Hundreds Almost non-stop operations (weekly maintenance window of about 1/2 day) Large Volumes of Data / High Volumes of Transactions Hundreds of Applications / Mission Critical Applications

MVS

17

Computing Environment - Typical IBM MF Site

Processing Environment

On-line during prime time (might mean 24 hours for global operations) Batch during non-prime time (wrt local time) of 12 - 15 hours

Software Environment

Variety of Databases / OLTP packages

EDI Processing
Two Tier Database Architecture - C/S and Central

Hardware Environment

Multiple Machines - Networked Together Multiple Processors for Each Machine Huge Number of Data Storage Devices - Disks and Tapes

MVS

18

Computing Environment - Typical IBM MF Site

Support Environment

Huge IT Departments Application Programming Staff Development Maintenance / Support

DBAs Operations Multiple Data Centers to Manage Batch Processing System Programmers for; - O/S - Database Packages - OLTP Packages Network Support Staff
MVS

19

IBM Families of Operating Systems

DOS - Disk Operating System For Small System/360 DOS/VSE (Virtual Storage Extended) VM/CMS - Virtual Machine Conversation Monitor System Simulates more than one computer system on single real machine Supports both DOS and OS OS - Operating System For Full range System/360 MVS (Multiple Virtual Storage)
MVS

Migrating from Dos to OS is a major change VM is not very popular Today most of the sites use MVS

20

MVS Evolution
1995 MVS/ESA 5.2.2

1993
1990 1988 SYSTEM 390

MVS/OPEN EDITION(POSIX)

MVS/ESA

16 TB

1981
1974 1972 OS/VS1

MVS/XA

2 GB (31-bit)

OS/VS2R2(MVS) 16 MB (24-bit) OS/VS2R1(SVS)16 MB

1970
1966 1966

SYSTEM 370
OS/MFT OS/MVT 3 MB

PRIMARY CONTROL PROGRAM (PCP)

MVS

21

MVS Evolution
Operating Systems for System/360 1965 Basically a Primer Introduction of JCL-(JOB Control Language) (Became obsolete 1n 1969 1966 Limited multi-programming Small improvement over PCP 1967/68 MFT
PCP

Primary Control Program

No Multiprogramming No Priority No Spooling Very unreliable Multi-programming with Fixed number of Tasks Fixed Number of Partitions One Job per Partition Fixed Partition Size

MVT

Multi-programming with Variable number of Tasks

Storage allocated as per requirement (region) No. of Jobs depends upon storage req. and availability 22

Reliable Comprehensive features Predecessor of MVS

MVS

MVS Evolution
Major Handicaps Limited and inefficient spooling No Virtual Storage

Utilities to Overcome these Handicaps HASP - Houston Automatic Spooling Priority

Developed unofficially (self initiative) by IBM employees Distributed free to MVT/MFT users Became very popular Eventually owned and supported by IBM

MVS

23

MVS Evolution
ASP - Attached Support Processor

Developed (officially) by IBM Intended for MVT Several mainframes can work together under single O/S (predecessor of multi-processing?) Provided better spooling capability Relatively less takers

MVS

24

MVS Evolution
System 370
Announced in early 70s Supported Virtual Storage New Operating Systems OS/VS were introduced

OS/VS1 (Virtual System 1) - adopted from MFT OS/VS2 (Virtual System 2)

Version SVS - Single Virtual Storage - adopted from MVT (1972) Version MVS - Multiple Virtual Storage - Completely Rewritten (1974)

HASP and ASP were migrated to OS/VS2 under the names JES2 and JES3 MVS and its derivatives are the mainstay of IBM O/S now
MVS

25

MVS Evolution
1974 24 bit addressability Greater reliability / integrity / performance Virtual storage limitation of 16MB - A serious handicap for CICS 1983 32 bit addressability Virtual storage of 2GB Increased maximum number of channels and Devices 1988 Virtual storage of 16 Terra Bytes OS /VS2 MVS/SP1 Multiple Virtual Storage System/370

MVS/XA Processors : 308x, 3090, 4381 MVS /ESA MVS/SP3 Multiple Virtual Storage Enterprise Systems Architecture ESA/370 Processors : ES/3090 E and S Models and later 4381 Models 90E, 92E, 93E and later

1990 Virtual storage of 16 Terra Bytes

MVS /ESA

MVS/SP4 Multiple Virtual Storage Enterprise Systems Architecture ESA/390 Processors : ES/9000, ES/3090-9000T ES/3090 J Models

MVS

26

The Von Neumann Computing Model

Most

common model for computing systems by John Von Neumann in year 1940

Prepared

Instruction 1

Instruction 2
Instruction 3 Instruction 4 ---- ----- ---Instructions are executed one at a time sequentially
MVS

Register

27

Operating System Considerations

a) Process Management Problem :- According to Jon Von Neumann model only one instruction gets executed at a time. What will happen if that instruction is waiting for I/O. In this case CPU time is wasted. Solution :P1 P2 P3 3) Now this will get executed 1) executing instruction 2) I/O operation READ

File Auxiliary Device

Hard Disk Memory

Main Memory

This gave rise to multiprogramming

MVS

28

Operating System Considerations

b) Memory Management Problem :- Any thing that is to be executed, must be in memory. (memory limitation) Solution :- 1. Place task in real memory 2. Place task in virtual memory 1. Real memory implementation : - code & data are in real memory - size of code & data limited by size of installed memory - good performance, low overhead - possible wastage of memory 29

MVS

Operating System Considerations

Memory Management (Contd ..) 2. Virtual memory implementation : based on assumption for a task, not all code & data is needed in real memory all the time implemented on a combination of real plus auxiliary storage operating system takes responsibility of bringing rest part of tasks in real memory when required.

MVS

30

Operating System Considerations

swapped out P1 P2 swapped in P1 P2 DATA

Perform I/O operation

P3
Virtual Memory

OS

Real Memory (Only part of program that requires main memory will be in the main memory)

Advantage : code and data size independent of the real memory

MVS

31

Operating System Considerations

c) Input-output Management Problem :- Application should not worry about device characteristics. I/O device speed is 100 times slower than CPU. Solution : Let all I/O be handled by a specialized systemI/O Subsystem

MVS

32

Operating System Considerations

c) Input-output Management ( Contd.. )
2) I/O operation swapped out P1 P2 P3 How it is to be done is decided by I/O subsystem Virtual Memory OS Real Memory Tells What to do P1 P2 I/O Subsystem

swapped in

File

Auxiliary Device

MVS

33

System 370 I/O Architecture

Channel 0 Channel 1 Channel 2 Channel 3 CPU Main Storage Channel 4

Channel 5
Channel 6 Channel 7

MVS

34

System 370 I/O Architecture

Typical Processor Configuration
Processor 4381 3084 3090 ES/9000 CPUs 1 or 2 4 6 4-8 Main Memory Maximum Channels 4 - 6 MB 18 32-128 MB 48 64-512 MB 2048 MB 128 256

MVS

35

System 370 I/O Architecture

STORAGE CP CP Channels

Devices

Channels Provide paths between the processor & I/O devices 3090 processors can have a maximum of128 channels A channel itself is a computer & executes I/O instructions called channel commands I/O devices are connected to channels through an intermediate device called Control Unit. Each channel can have up to 8 control units. 36 MVS

System 370 I/O Architecture

Control Unit : These are DASD units, can be connected to common control unit, called String Controller. String Controller can be connected to a channel directly or indirectly A control unit called Storage Control connects string controllers to a channel.
channel connection (1 - 128) storage control

string controller 1 (1 - 8) string controller 1 DASD unit MVS

32

32

37

IBM Hardware
How do todays PC and medium sized IBM MF compare?
Characteristics Processor Speed Main Memory Individual Disk Storage Monitor PC (Pentium 100) 16-32 MB 16-32 MB 1.2 GB SVGA/ Graphics Main Frame (4381) 32 MB 32 MB 946 MB Character based dumb terminal

MVS

38

IBM Hardware
Where does the power of IBM MF come from? Multiple processors with partitioning capability Cache memory and expandable memory Multi-user / Multi-programming Support Batch and on-line processing support Local and remote terminal support High number of devices Strong data management capability

MVS

39

Key Terminology
Cache Memory High speed memory buffer (faster than main memory) Operates between CPU and main memory Used to store frequently accessed storage locations (instructions) Usually available on all processors Expanded Memory Supplements main memory Not directly available to application program Not directly accessible by CPU Implemented using high speed disk Usually available with higher-end machines 40 MVS

Key Terminology
Processor Resource / System Manager (PR/SM)

Used to control Multi-processor Configurations Allows division of multi-processors in partitions - LPAR Each partition functions as independent system Enables fault tolerance implementation by designating Primary and Secondary Partitions Secondary partition takes over automatically if primary fails Allows reconfiguration of I/O channel to partitions

MVS

41

Key Terminology
Channels Device Management Concept - Unique to IBM Provides access path between CPU and I/O devices (DMA) Upto eight control units can be connected to one channel Upto eight I/O devices can be connected to one control unit A channel is a small computer in itself with a set of instructions (Channel commands) Channel controls the I/O device operations independent of CPU Cannel processing can overlap CPU processing improved performance 42 MVS

Key Terminology
Channel - I/O Device Connectivity Parallel architecture i.e. all bits of a byte are transmitted simultaneously Information transfer is in unit of two bytes Sixteen data wires and additional control wires are required Maximum length of 120 meters (400 feet) Data speed of 4.5mbps Use of copper results in heavy, expensive cabling ESCON - Enterprise System Connection Announced in 1990 Uses fiber optic Results in reduced size and weight Length limit extended to approximately 42Km (26 miles) Faster data speed (17mbps)
MVS

43

I/O Devices
Unit Record Devices - Each record is a single physical unit Card Devices (now obsolete) : Readers / Punches / Reader and Punches Printer

Impact Printers - 600 to 2000 LPM Non-Impact Printers - 3800 sub-system, 20,000 LPM

Built-in control units for each device Directly attached to channel

Magnetic Tape High volume storage Sequential processing Normally used as back-up device Also used for physical transfer of data 4 to 8 tape drives are connected to one control unit
MVS

44

I/O Devices
DASD - Direct Access Storage Device IBMs official name for Disk Non-removable - offers better reliability and are faster Each unit is called as disk pack or Volume Each pack has multiple surfaces Each surface has multiple tracks Same track no. of all surfaces together constitute a Cylinder DASD capacity ranges from 100 MB (3330) to 8514MB (3390/9) A group of DASDs of same type are connected together to form a String and are connected to a string controller Multiple string controller are connected to a storage controller Storage controller is connected to channel 45 MVS

I/O Devices
3990 Storage Controller Can connect 2 strings of 32 each 3390 model DASDs (totally 64 DASDs Consists of high speed cache storage (32MB to 1024MB) Data is buffered using cache Frequently accessed data is stored in Cache - improved performance Supports more than 4 channel connection to processor

Enables several simultaneous disk operations

MVS

46

Data Communication Equipment

Data Communication equipment lets an installation create a data communication network that lets users at local terminals & remote terminals to access computer system

MVS

47

Data Communication Equipment

Allows local and remote terminals access to the computer systems
Components of data communication Host Computer - System/370 processor Communications Controller - Attached to the channel Devices (terminals and printers) are connected to the terminal controller (also known as cluster controller) Terminal controller is connected to communications controller Terminal Controller managing Local terminals / printers can be connected directly to the channel 48

MVS

Data Communication Equipment

Modems and telecommunication lines (telephone line,

Satellite Link) Remote terminals / printers are connected to terminal controller (at local site) Terminal controller is connected to modem Modem is connected to telecommunications line At the receiving end telecommunications line is connected to modem Modem is connected to communication controller

MVS

49

Data Communication Equipment

3270 Information Display System Sub-system of terminals, printers and controllers connected to Host computer Locally through communications controller or directly to channel Remotely through communications controller, modem and telecommunications line A typical 3270 terminal controller (3274) controls upto 32 terminals / printers Emulator programs (Shine Link, Erma Link) allow computers (typically PCs) to mimic 3270 devices These are useful since they allow upload / download of data between MF and PC
MVS

50

Data Communication Equipment

System/370 processor Host System

channel
communication controller

modem

modem

Telecommunication Lines
modem modem

terminal controller

terminal controller

terminal

printer

terminal

printer

Remote System MVS

Remote System

51

Data Communication Equipment

At the center of the network is the host system, a system/370 processor The control unit that attaches to the host systems channels is called a communication controller it manages the communication function necessary to connect remote terminal system via modems and telecommunication lines A modem is a device that translates digital signals from the computer equipment at the sending end into audio signal that are transmitted over a telecommunication line, which can be telephone line, a satellite link or some other type of connection At the receiving end of the line, another modem converts those audio signals back into digital signal 52 MVS

Often Asked Questions About IBM

Why is grasping IBM difficult? User interface is poor

Non-intuitive Unfriendly and Formidable

Ancient terminology (e.g. Card, Punch queue) which is irrelevant now Different terminology (e.g. DASD, DATA SET) Too many options / parameters Too many terms / acronyms Variety of software results in site specific variations

MVS

53

Often Asked Questions About IBM

Why is IBM so Popular Sturdy and Secure HW/SW Downward compatibility (does not make application SW obsolete) Excellent customer support Variety of software (Databases, OLTP packages) - IBM and Third Party Market Leader

First in DBMS, First in OLTP, first in PC!!!! First to develop a chess playing computer that beat world champion

MVS

54

Often Asked Questions About IBM

The old legend : Nobody got fired for buying IBM Future of IBM (is there any?) Large existing application base will need support Downsizing will need knowledge of current application/platform Dual skills will be much in demand Not all applications are suitable for downsizing - many will remain on MF MF will be increasingly used as back-end server New applications (data warehousing type) will be developed on MF Multi-tier architecture will become common Bottom Line : It is too soon to herald death of IBM MF
MVS

55

Characteristics Features Of MVS

1] VS : The use of virtual storage increases the number of storage locations available to hold programs and data 2] MULTIPROGRAMMING : Multiprogramming simply reclaims the CPU during idle periods to let other programs execute 3] SPOOLING : To provide shared access to printer devices, spooling is used 4] BATCH PROCESSING : When batch processing is used, work is processed in units called Jobs. A job may cause one or more programs to be executed in sequence. Batch jobs get collectively processed by the system
MVS

56

Characteristics Features Of MVS

5] TIMESHARING : In this system, each user has access to the system through a terminal device. Instead of submitting jobs that are scheduled for later execution, the user enters commands that are processed immediately.
Time sharing is also called as Online Processing because it lets users interact directly with the computer.

MVS

57

MVS Terminology
Address Space An address space is simply the complete range of addresses and as a result, the number of storage locations; that can be accessed by the computer. An address space is a group of digits that identify a physical location in main storage In MVS an address space has 24-bit positions i.e. 16MB addressability. MVS allows each programmer to use all 16MB address space, even though real storage includes only, for example 4MB physical locations.
MVS

58

MVS Terminology
Address Space (Contd..) In MVS, references in the program address space are not associated with a particular real storage location. They remain reference to a particular piece of information called Virtual Addresses. They become real only when assigned to a physical location. When the program is ready to execute the system, using a system/370 hardware feature called Dynamic Address Translation(DAT), maps the virtual addresses in the program to the real storage addresses. By doing this, MVS can make the program address space larger than the number of physical location available in real storage. 59 MVS

MVS Terminology
MVS It uses real storage to simulate several address spaces, each of which is independent of the others

Address Space I Real Storage II III

Auxiliary Storage 0 0 0

MVS

60

MVS Terminology
Auxiliary storage and real storage are used in combination to simulate several virtual storage address space Each batch job or TSO user is given its own address space Various factors such as the speed of the processor and the amount of real storage installed effectively limit the number of address spaces than can be simulated. To provide for the larger virtual storage, MVS treats DASD as an extension of real storage Only one address space can be in control of CPU
MVS

61

MVS Terminology
Paging To enable the movement of the parts of a program executing in virtual storage between real storage and auxiliary storage, the MVS system breaks real storage, virtual storage & Auxiliary storage into blocks. - A block of Real Storage is a Frame - A block of Virtual Storage is a Page - A block of Auxiliary storage is a Slot A page, a frame and a slot are all the same size each is a 4K byte An active virtual storage page resides in a real storage frame, an inactive virtual storage page resides in an auxiliary storage slot 62 MVS

MVS Terminology
Paging Moving pages between real storage frames and auxiliary storage slots is called PAGING
Virtual Storage Page

Frame Real Storage PAGING

Slot Auxiliary Storage

MVS

63

MVS Terminology
Demand Paging Assume that DAT encounters an invalid page table entry during address translation, indicating that a page is required that is not in a real storage frame. To resolve this Page Fault, the system must locate an available real storage frame to map the required page(page-in). If there is no available frame, an assigned frame must be freed. To free a frame, the system moves its contents to a auxiliary storage. This movement is called a Page-Out. System performs page-out only when the contents of the frame have changed since the page was brought into real storage.

Once a frame is located for the required page, the contents of the page are moved from auxiliary storage to real storage. This movement is called as Page-In.
MVS

64

MVS Terminology
Demand Paging (Contd ..) The process of bringing a page from auxiliary storage to real storage in response to a Page Fault is called DEMAND PAGING

MVS tries to avoid the time consuming process of demand paging by keeping an adequate supply of available real storage frames constantly on hand. Swapping is one means of ensuring this adequate supply. Page stealing is another.

MVS

65

MVS Terminology
Swapping Swapping is the movement of an entire address space between Virtual storage & Auxiliary storage. It is one of the several methods MVS employs to balance system workload, as well as to ensure that an adequate supply of available real storage frames is maintained. Address space that are swapped in are active, having pages in real storage frames & pages in auxiliary storage slots. Address spaces that are swapped out are inactive; the address space resides on auxiliary storage and cannot execute until it it is swapped in.
MVS

66

MVS Terminology
Page Stealing If there are not enough 4K frames available then frames which are not referenced for long time will get thrown out and written to the auxiliary storage. So now those 4K frames are free. This is known as Page Stealing.

MVS

67

MVS Terminology
The paging process is managed by several components of MVS. The 3 major one are : - Real Storage Manager (RSM) - Auxiliary Storage Manager(ASM) - Virtual Storage Manager (VSM) RSM - manages real storage - directs movements of pages among real and auxiliary - builds segment & page table
MVS

68

MVS Terminology
ASM keeps track of the contents of the page dataset and swap dataset page dataset contain virtual pages that are currently occupying a real storage frame. Swap dataset contain the LSQA pages of swapped out address space. VSM controls allocation/deallocation of virtual storage maintain storage use information for Storage Management Facility (SMF)
MVS

69

Virtual Storage Layout

Common Area System Queue Area (SQA)

Pageable Link Pack Area (PLPA) Common Service Area (CSA)

Private Area

Users Private Address Space

Local System Queue Area (LSQA)

Scheduler Work Area (SWA)

User Region

System Area

Nucleus

System Region

Each Virtual Storage Address Space consists of a System Area, a Private Area and a Common Area.
MVS

70

Virtual Storage Layout

System Area It contains the nucleus load module, page frame table entries. Data blocks for system libraries and so many other things Nucleus and other contents of the System Area make up the resident part of the MVS system control program Its contents are mapped one for one into real storage frames at initialization time. The size of System Area does not change once it is initialized

MVS

71

Virtual Storage Layout

Common Area

It contains parts of the system control program, control blocks, tables and data areas The basic parts of the Common Area are :
System Queue Area (SQA) Pageable Link Pack Area (PLPA) Common Service Area (CSA)

MVS

72

Virtual Storage Layout

System Queue Area
contains tables and queues relating to the entire system

the contents of SQA depends on an installations

configuration & job requirement

It is allocated from the top of the virtual storage in 64K

segments, a minimum of 3 segments are allocated during system initialization.

Allocated SQA space is both non-swappable and non-

pageable
MVS

73

Virtual Storage Layout

Pageable Link Pack Area - Contains svc routines, access methods, other system programs, and selected user programs. - It is pageable - Because the modules in PLPA are shared by all users, all program modules in PLPA must be reentrant and read-only - PLPA space is allocated in 4K block directly below SQA. - The size of PLPA is determined by the number of modules included - Once the size is set, PLPA does not expand

MVS

74

Virtual Storage Layout

Common Service Area Contains pageable system and user data areas. It is addressable by all active virtual storage address space

and shared by all swapped-in users.

Virtual storage for CSA is allocated in 4K pages directly below PLPA.

MVS

75

Virtual Storage Layout

Private Area The Private Area is made up of : - Local System Queue Area(LSQA) - Scheduler Work Area(SWA) - Subpools 229/230 - System Region - User Region The user region is the space within Private Area that is available for running the users program Local System Queue Area - LSQA contains tables and queues that are unique to a particular address space
MVS

76

Virtual Storage Layout

Private Area (Contd ..) Scheduler Work Area

- SWA contain control blocks that exist from task initiation to task termination - The information in SWA is created when a job is interpreted and used during job initiation and execution - It is pageable and swappable

MVS

77

Data Management Overview

Anything that needs to be stored and accessed on user request is a Data for MVS Types of Data Business Data Database Indexed Files Flat Files Application Components Source Programs Executable Programs Screen Definitions Record Layout Definitions Command File Scripts
MVS

78

Data Management Overview

Types of Data (Contd ..) MVS (System) Data O/S program User Information (ID, Password, Profile) Access Permissions Temporary Data O/S Built Data (e.g. task queues, segment table, page table) Spooled Output Work Files for Sort

MVS

79

Dataset Organization
Dataset organization fall into two categories under MVS : VSAM and NON-VSAM Non-VSAM provides four basic ways of organizing data stored in datasets - Physical Sequential - Indexed Sequential - Direct - Partitioned VSAM provides four basic ways of organizing data stored in datasets - Entry Sequence Dataset - ESDS - Key Sequence Dataset - KSDS - Relative Record Dataset - RRDS - Linear Dataset - LDS 80 MVS

Data Management Overview

Non-VSAM datasets organization
Physical Sequential Records are stored one after another in consecutive sequence Can reside on just any type of I/O devices Appropriate when files records dont have to be retrieved at random Indexed Sequential Includes an index, which relates key field values to the location of their corresponding data records

MVS

81

Dataset Organization
Non-VSAM datasets organization
Direct Permits random access of records It doesnt use an index To access record, the disk location address (By hashing) of that record to be specified

Partitioned Consists of one or more members Each of these members can be processed, as if it were a separate physical sequential file. Names of members in a Partitioned dataset(PDS) is stored in a directory
MVS

82

Dataset Organization
Partitioned Data Set - Salient Features Commonly referred as PDS Also known as Library Used to store application components PDS is divided into one or many members Member name can be up to 8 characters long There is no extension for member Each member can be processed as an individual unit Entire PDS can be processed as one unit Each PDS contains a directory

MVS

83

Dataset Organization
Partitioned Data Set - Salient Features (Contd ..) Directory has an entry for each member in a PDS

PDS Examples PAYROLL.TEST.SOURCE, PAYROLL.PROD.SOURCE, INV.TEST.LOADLIB Normally consists of 3 qualifiers called as PROJECT GROUP TYPE

MVS

84

Dataset Organization
Partitioned Data Set - Salient Features (Contd..) Personal PDS start with high level qualifier as User ID E.g. DA00T23.NEW.SOURCE

Member Name Examples PAB0017, PAB0105, PAC0021 etc. Usually, the application component type cannot be identified from the member name. For that naming conventions are used for PDS.
MVS

85

Dataset Organization
VSAM datasets organization
ESDS Can only reside on DASD Functionally equivalent to Physical Sequential File KSDS Functionally equivalent to Indexed Sequential File RRDS Lets you retrieve the record by specifying the location relative to the start of the file All VSAM datasets must be cataloged
MVS

86

Dataset Organization
Data Organization - Salient Points Non-VSAM Data Organization was developed in mid 1960s VSAM - (Virtual Storage Access Method) was introduced in early 1970s VSAM was expected to replace Non-VSAM Data Organization Functions Today, most sites use both VSAM and Non-VSAM Data Organization VSAM is the primary data organization for user data VSAM is also called as native file management system of IBM Most of the DBMS running under MVS use VSAM as underlying Data Organization (e.g. DB2, IDMS) 87 MVS

Dataset Organization
Data Organization - Salient Points (Contd ..) Physical Sequential Data Organization is used for flat files Index Sequential and Direct Data Organization are not very popular now (these functions are handled better by VSAM) Partitioned Data Sets (PDS) also used by MVS to store O/S programs

MVS

88

Data Set Naming Rules

Data Set Naming Rules Allows - Alpha, Digits, National Characters @,#$, and . Maximum Length 44 characters for DASD, 17 for Tape If Length is more than 8, must be broken into qualifiers of maximum 8 characters each Qualifiers are separated by . . are counted in overall length First character of the qualifier must be alpha or national character Last character of data set must not be . First qualifier is called as high-level qualifier High-level qualifier has special significance
MVS

89

Data Set Naming Rules

E.g. Data Set name PAYROLL.P9710.TRAN Has three qualifiers High-level qualifier is PAYROLL Total length is 18

MVS

90

Data Set Tracking

Data Set Tracking Mechanisms

Label Catalog

Label Data Set Label

First record of each data set is a label record called as; File label or Data Set Control Block (DSCB) There are several formats for DSCB DSCB describes data sets name, its DASD location and other details

MVS

91

Data Set Tracking

Label (Contd..) DASD Label

Each DASD is labeled; called Volume Label (VOL1 label) DASD label is stored on a disk at third record of track 0 in cylinder 0 DASD label contains Volume Serial Number and address of the VTOC file

Volume Serial Number

Each DASD is identified by a unique number, Volume Serial Number vol-ser Vol Ser must be specified for accessing the Data Set (which is not cataloged)

MVS

92

Data Set Tracking

Label (Contd..) VTOC VTOC - Volume Table Of Contents is a special file for each DASD VTOC contains the file labels for all data sets on the volume

MVS

93

MVS Datasets
Label Processing When a dataset is stored on disk or tape, MVS identifies it with special records called labels. There are 2 types of DASD labels : Volume, File Label All DASD volumes must contain a volume label, often called a VOL1 label. This label is always in the same place on a disk volume : the 3rd record of track zero in cylinder zero. Volume label has 2 important functions : - It identifies the volume by providing a volume serial no. : Vol-ser. Every DASD volume must have a unique sixcharacters vol-ser. - It contains the disk address of the VTOC.
MVS

94

MVS Datasets
Label Processing (Contd..)

The VTOC (Volume Table of Contents) is a special file that contains the file labels for the datasets on the volume.
These labels are called Data Set Control Block (DSCB) have several formats called Format-1, Format-2 and so on.

MVS

95

MVS Datasets
VOL 1 Label Vtoc Format-4 VTOC Free Extents File-A File-B File-C Free Extent File-B Free Extent Format-1 FileA Format-I File-B Format-I File-C Format-3 File-B Format-5 Free Extents

MVS

96

MVS Datasets
Format-4-dscb : describes VTOC itself Format-1-dscb : *describes a dataset by supplying dataset name, DASD location & other characteristics [space is not allocated to DASD file in area called extents. Each extent consists of one or more adjacent tracks] : * has room to define 3 extents for a file (1 primary, 2 secondary) Format-3-dscb : *if file requires more than 3 extents, this dscb is created *It contains room for 13 additional secondary extents [As a result file can contain upto 16 extents] Format-5-dscb : *contain information about free extents that arent allocated to files * each can define upto 26 free extents

MVS

97

Catalog
Catalog

Obviates the need of specifying Vol Ser for the data set

Catalog Types

Master Catalog User Catalog

Catalog Features

Each MVS has only one Master Catalog Master Catalog is used by MVS for system data sets User Catalog is used for user data sets There can be multiple User Catalogs Master Catalog contains one entry of each User Catalog

MVS

98

Catalog

Catalog Features (Contd..) VSAM data sets must be Cataloged Non-VSAM Data Sets may or may not be cataloged An Alias can be created for a Catalog Usually, the high-level qualifier of a data set is same as the catalog name or catalog alias name Multiple data sets can be cataloged in single user catalog Alias helps to catalog data sets with different high-level qualifiers to be cataloged in a single user catalog
Catalog Name VCAT.MPS800 Alias MMA2 MMA1

Data Set Name MMA2.CUSTOMER.MASTER MMA1.CUSTOMER.TRAN MVS

99

Data Management
Data Management Functions (for Non-PDS) Allocate Process Add Records Modify Records Delete Records Deallocate (delete) Copy Rename Catalog

MVS

100

Data Management
Additional Functions for PDS

Compress Member Management Create, Modify, Delete, Copy, Rename

How Data Management is Achieved

Interactively using MVS Commands Executing MVS Utility Programs (batch mode) Through Application Programs On-line Processing Batch Processing
MVS

101

MVS Concepts
How datasets are Accessed Generally dataset goes through three phases when handled through program : Allocation Processing Deallocation Allocation The process of locating an existing dataset or space for a new dataset and preparing the system control block needed to use the dataset is called Allocation Allocation occurs at 3 levels
Unit is selected and allocated e.g. SYSALLDA-DASD, TAPE Volume is allocated Dataset on that volume is allocated
MVS

102

MVS Concepts
How datasets are Accessed (Contd ..)

Processing - Processing involves 3 steps

Opening datasets Processing I/O Closing datasets

Deallocation - Each file is automatically deallocated when job is finished with it - While deallocating, disposition of dataset can be decided, whether you want to retain the file or should be deleted
MVS

103

Job Management Overview

What is a Job? Simply put, job is execution of one or more related programs in sequence E.g. 1 A job of creating an executable module (load module) from a source program consists of executing Compiler program and executing Linker program. E.g. 2 A job of printing invoices may consist of execution of three programs; an EXTRACT program to pull out transactions from database, a SORT program for sorting the transactions, a PRINT program to print the invoices.
MVS

104

Job Management Overview

Job - Salient Points Executed in a background mode Job details are specified using some command language Job Management Functions Receive the job into operating system Schedule the job for processing by O/S Execute the Job Process the output

MVS

105

Job Management Overview

Stages of Job 1. Job Preparation User keys-in commands using Editor Save as a member in PDS 2. Job Scheduling Initiated using TSO SUBMIT command Not necessarily on FIFO basis Prioritization is implemented using concept of class and priority code 3. Job Execution

MVS

106

Job Management Overview

Stages of Job (Contd..) 4. End of execution (normal, erroneous) Intimate the user Job log management Job output management Printer output Data set output

MVS

107

Job Management Overview

Erroneous Termination of job Type of execution errors

Incorrect commands (command syntax errors) Required resources (Data Sets, Program Library, Program Load Module) not available Violation of access permissions for data sets, program load module etc. Mismatch in data set status; as required by job and as it actually exists e.g. a create is issued for a data set while data set which already exists

MVS

108

Job Management Overview

Erroneous Termination of job (Contd ..)
Program errors Mismatch for Data set - Between program definition and actual characteristics Infinite loop Data Type mismatch - numeric variable contains nonnumeric data Any abnormal termination of program is called as Abend

MVS

109

Job Management
Definitions JOB :- Is the execution of one or more related programs in sequence JOB STEP :- Each program to be executed by a Job is called a job step

JCL (Job Control Language) :- Is a set of control statements that provide the specifications necessary to process a job

MVS

110

Job Management
Definitions (Contd ..)
JES (Job Entry Subsystem) :

- Ment for job entry into system, also for job returning after completion - Shares the load on the operating system - Takes care of all inputs and outputs - Do simple basic syntax checking - Resource Initialization - Creation of address space - It is also known as Job Scheduler - Classified into JES2-design for uniprocessor environment JES3-design for multiprocessing environment (Decided at the time of system initialization) - Jobs are sent to MVS depending on the class priority schemes 111 MVS

How Job Is Entered Into the System

When you submit the job, JES reads the job stream(sequence of JCL commands) from a DASD file and copy it to a job queue, which is a part of a special DASD file called JES SPOOL.

MVS

112

Job Scheduling
How Job Is Scheduled For Execution
MVS does not necessarily process jobs in the order in which they are submitted. Instead, JES examines the jobs in the job queue and selects the most important jobs for execution. That way JES can prioritize its work, giving preference to more important jobs. JES uses 2 characteristics to classify a jobs importance, both of which can be specified in the jobs JCL : Job Class and Job Priority If two or more jobs are waiting to execute, the JES scheduler selects the one with higher priority 113

MVS

Job Scheduling
Each job class is represented by a single character, either a letter (A-Z) or a digit (0-9). Job classes are assigned based on the processing characteristics of the job.
INITIATOR :- An initiator is a program that runs in the system region of an address space. Each initiator can handle one job at a time. It examines the JES spool, selects an appropriate job for execution, executes the job in its address space and returns to the JES spool for another job. The number of active initiator on a system and as a result the number of address spaces eligible for batch job processing determines the number of batch jobs that can be multiprogrammed at once.
MVS

114

Job Scheduling
Each initiator has one or more job classes associated with it. It executes jobs only from those classes. Initiator 1 2 3 4 5 6 Eligible Job Classes A B,C,D,H,L,T B,C,D,H,L,T B,C B,C C

MVS

115

Job Scheduling
Within a job class, initiator selects jobs for execution based on their priorities, which can range from 0 to 15
If two or more jobs have same class & priority, they are executed in the order in which they are submitted.

MVS

116

How Job Is Executed

Once an initiator has selected job for execution, it invokes a program called the interpreter The interpreter's job is to examine the job information passed to it by JES and create a series of control blocks in the SWA, a part of the address spaces private area Among other things, these control blocks describes all of the datasets the job needs Now initiator goes through 3 phases for each step of job
- Allocation (required resources are allocated) - Processing (region is created & program is loaded and executed) - Deallocation (resources are released)

This continues until there are no more job steps to process

117

MVS

How Job Is Executed

This continues until there are no more job steps to process Then, the initiator releases the job and searches the spool again for another job from the proper class to execute As a users program to execute, it can retrieve data that was included as part of job stream and stores in the JES spool

Job JES

spool

INITIATOR

JCL SYSOUT QUEUE SYSOUT O/P DATA

SYSIN PROGRAM EXECUTION INTERPRETOR

MVS

118

How The Jobs Output Is Processed

Like Jobs, SYSOUT data is assigned an output class that determines how the output will be handled Common O/P classes are A :- Printer B :- Card Puch O/P X :- Held O/P [Held O/P stays on the sysout queue indefinitely; Usually, O/P is held so that it can be examined from a TSO terminal]

MVS

119

How A Job Is Purged

After the jobs output has been processed, the job is purged from the system, i.e. JES spool space, the job used, is freed so it can be used by other jobs and any JES control blocks associated with the job are deleted
Once a job has been purged, JES no longer knows of its existence

MVS

120

Dataset Allocation And Job Step Execution

SWA control block System Region Interpreter Initiator SWA control block

After initiator selects a job for execution, it invokes the interpreter, which builds the required control blocks in the address spaces SWA.

System Region Allocation Initiator SWA control block User Program System Region Initiator

For each job step, the initiator invokes allocation routines to allocate the units, volumes and datasets required by the job step

After resource allocation, the initiator creates a user region, loads the user program into it and transfers control to user program
MVS

121

Dataset Allocation And Job Step Execution

SWA

CONTROL BLOCK

SYSTEM REGION
UNALLOCATION
INITIATER

When the user program completes, the initiator invokes Unallocation routine to Deallocate the resources used by the job step 122 MVS

Tools Overview
Tools - A set of sub-systems and facilities that; Implement MVS functions Are directly used by MVS user

These are essentially Software programs; system programs

MVS

123

Tools Overview
Interactive Processing Tools TSO - Time Sharing Option Used by the terminal user to invoke MVS facilities interactively TSO internally treats each terminal user as a Job Job Stream is created when terminal user logs in Each terminal user is given a separate address space

MVS

124

Tools Overview
ISPF - Interactive System Productive Facility Runs as part of TSO Takes advantage of full screen (24 x 80) capability of 3270 terminals Panels are provided for terminal users for issuing commands Key Functions Implemented Using ISPF Editor - Program Sources, Job Commands Data Management - PDS and Physical Sequential Data Set Management Job Processing - Initiate Job, Check job log Miscellaneous PDF - Program Development Facility is Part of ISPF
MVS

125

Job Management Tools

Job Control Language (JCL) Used to code job commands Job Entry System (JES) Manages the job before and after execution; receive, schedule, process output Base Control Program (BCP) Manages the job during execution Simultaneous Peripheral Operations on-line (SPOOL) Used for staging of input and output

MVS

126

Job Management Tools

Why and What of JCL? JCL is the most dreaded word for newcomer to IBM world Why JCL? Since the job is executed in background, without user interaction, all information required for the execution must be supplied in advance JCL is used to specify this information

MVS

127

Job Management Tools

Why JCL?
The most common information supplied through JCL is;

To whom the job belongs (which user id)? What is the program / utility that is to be executed? Where (in which library / PDS) to find the load module of the program or utility? Where (which DASD volume / catalog, what data set name) to find the input data files for the program / utility? Where should (which DASD volume, what data set name) the output files be created? The printer output should be directed to which printer?

MVS

128

Job Management Tools

What is JCL?

Stands for Job Control Language Connotation is; set for job commands stored as a MEMBER in a PDS e.g. JCL to execute a batch program, JCL to compile and link a COBOL program, JCL to allocate a VSAM data set, JCL to SORT and MERGE two Physical Sequential Data Sets Thus, JCL is nothing but a set of commands User keys-in commands using a editor Saves as PDS Member e.g. PAYROLL.TEST.JCL(PROG1JCL)
MVS

129

Job Management Tools

What makes learning JCL so difficult JCL is powerful and flexible, that leads to some complexity It is non-intuitive The user interface is formidable The terms are ancient Very little has changed since 1965 when JCL was first developed However, JCL can be understood and mastered with logical approach and open mind Good grasp of JCL is a must to be a versatile IBM programmer
MVS

130

Jes - Job Entry System

Introduction

Two versions of JES; JES2/JES3

JES2 is primarily for single processor systems JES3 is for multiple processor systems

Each MVS system uses either JES2 or JES3 JES3 has additional scheduler functions than JES2 (e.g. schedule job at specific time of the day, interdependent job scheduling)

MVS

131

MVS Tools Overview

How Job Enters the System?
.Job can enter the system from local or remote card readers

(now obsolete)
.By starting a cataloged JCL procedure (e.g. when user logs

in, a predefined set of commands are executed as a batch job. These commands are stored as cataloged JCL procedure)
.By interactive users thru SUBMIT command. Users can

create a PDS member in which commands are specified. On issuing SUBMIT command these are executed as a job
We will focus on third approach
MVS

132

MVS Tools Overview

Input

On SUBMIT, internal reader reads the JCL and creates an input stream JES2 reads the input stream, assigns a Job Number and places input stream in SPOOL data set (a message is sent to TSO user about the job number) Job is put in the conversion queue

Conversion
Converter program analyzes JCL statements Converts into converter / interpreter text Checks for Syntax errors

If any error, Job is queued for output processing If no error, Job is queued for processing
MVS

133

MVS Tools Overview

Processing Selection based on job class and priority Selected job is passed to Initiator Initiator invokes Interpreter Interpreter builds control blocks from converter / interpreter text in a Scheduler Work Area (SWA)

SWA is part of address spaces private area Control blacks describe the data sets required by the job

MVS

134

MVS Tools Overview

Initiator allocates resources required by the Job

Initiator starts the program to be executed Builds the user region Loads the program in the user region Transfers control to the program

On completion of the program execution, initiator de-

allocates the resources The process of allocation / execution and de-allocation is repeated for each job step

MVS

135

MVS Tools Overview

Initiator Characteristics

Each initiator can handle one job at a time

There can be multiple initiators

Each initiator has a job class associated with it System Operators can control the number of initiators and the class/es associated with each initiator

MVS

136

MVS Tools Overview

Input Data

Input data to the users program can be specified in the job

Called as in-stream data or SYSIN data

SYSIN data is read and stored in JES spool SYSIN data is treated like a data coming from card reader

MVS

137

MVS Tools Overview

Output
Management of System Messages, User Data Sets that

need to be Printed / Punched

Organized by output class and device set-up requirements

User s program can produce output data that is stored in a

JES spool; called as SYSOUT data

Each SYSOUT data is assigned an output class

Output class indicates the printer selection

MVS

138

MVS Tools Overview

Output (Contd ..)
Held Output

Special class (usually Z) is assigned to hold the output Held output remains in the SYSOUT indefinitely Usually used to verify before printing User can change the class and thus release the held output

MVS

139

MVS Tools Overview

Hard-copy

Local or remote processing Device Selection Queue the output for print /punch

Purge

Release SPOOL and Job Queue space Intimate TSO user about job completion

MVS

140

MVS Tools Overview

Job Output Output is produced at each stage job processing Includes output produced by;

JES MVS Users program where SYSOUT is allocated as output device

Job output is available to user (you can see it dynamically) It can be viewed using ISPF

MVS

141

MVS Tools Overview

Components of Job Output Component 1- Separator Page

First and last page of job output Inserted by JES Helps operator the segregate the job outputs when directed to printer

Component 2 part I Job Log

Messages produced by JES Also displayed on operators console If the job Abends, error messages are logged in Job Log

MVS

142

MVS Tools Overview

Component 2 part II Job Statistics

Summary information of system resources used by the job e.g.; Number of JCL cards Spool usage Execution time

Component 3 - JCL Listing List of JCL that was processed for the job Should be same as what user has created

MVS

143

MVS Tools Overview

Component 4 - Message Log
Messages regarding job execution Messages produced by MVS Includes details of Resource Allocation Program Execution Resource De-allocation Consists of Message label and message text Message label starting with IEF indicates a MVS message Installation specific messages

Component 5 - SYSOUT Separate sub-component for each SYSOUT allocation Each SYSOUT can have different characteristics e.g. class, record length etc.
MVS

144

MVS Tools Overview

VTAM Virtual Telecommunications Access Method Telecommunications (TC) Access Method Required to support terminal devices Part of SNA System Network Architecture Provides centralized control over all terminals attached to the system VTAM Application programs (e.g. TSO, CICS IMS-DC) communicate with terminal devices via VTAM

MVS

145

MVS Tools Overview

CICS Customer Information Control Program (optional component)

Interactive applications are developed using CICS CICS is a VTAM application program Works with VTAM to support on-line functions CICS implements multi-programming within itself Multiple programs which are part of same application are executed within CICS address space CICS selects one program at a time for execution CICS itself is multi-programmed by MVS along with other programs

MVS

146

MVS Tools Overview

DB2 - DataBase 2 (optional component) Database Management System Relational Implementation RACF - Resource Access Control Facility
Comprehensive Security Package Though optional used by most of the installations Users and Resources (e.g. Data Sets) are identified to

RACF Whenever user tries to access a resource the security is checked by RACF RACF is a set of routines Invoked as and when required
MVS

147

MVS Tools Overview

SMF - System Management Facility

Keeps track of system usage CPU, DASD I/O, Records Printed etc. Data collected when job is executed Stored in a special data sets Used for billing

MVS

148

MVS Tools Overview

Language Translators / Linkage Editor / Loader Language Translators- Convert source to object module Separate for each language, Assembler Language Translator is part of MVS Linkage Editor (part of MVS) - Converts object module to executable I.e. load module Loader - Creates temporary load module (used during testing phase)

MVS

149

MVS Tools Overview

Utilities
Set of general purpose programs Executed like a user program through JCL

Common Utilities are :

IEBGENER IEFBR14 SORT IDCAMS

MVS

150

Assignments on ISPF & TSO

1. Create 2 PDSs with high level qualifier as your own MF login-id
2. Create 1 flat file EMPLOYEE with employee details like emp-no,
emp-name, dept, desig, salary

3. See your created PDSs and file in hte dataset list

4. See, Browse, Edit the members of a PDS.

5. Copy 3 members from one PDS to another 6. Move one of the members from one PDS to another 7. Rename the moved member 8. Rename the members at the time of copying and moving 9. Write a sample COBOL program to display the hard coded values. 10. Make the proper alignment for the same program

11. Transfer the given member JCL to other user.

12. Delete the least important member from your PDS. Delete the memberless PDS. MVS

151