Security Analysis and Enhancements of Computer Operating Systems
Author
R. P. Abbott, J. S. Chin, J. E. Donnelley, W. L. Konigsford, S. Tokubo, D. A. Webb
Abstract
The protection of computer resources, data of value, and individual privacy has motivted a concern for security of EDP installations, especially of the operating systems. In this report, three commercial operating systems are analyzed and security enhancements suggested. Because of the similarity of operating systems and their security problems, specific security flaws are formally classified according to a taxonomy developed here. This classification leads to a clearer understanding of security flaws and aids in analyzing new systems. The discussions of security flaws and the security enhancements offer a starting reference for planning a security investigation of an EDP installation\'s operating system.
Organization
U.S. Department of Commerce
Affiliation
Institute for Computer Sciences and Technology
Publication Date
2001-01-01
Contents
Abstract..........................................1
1. An Overview..............................1
1.1 Motivation for Enhancing
Security.................................2
1.2 Technical Issues in
Enhancing Security.............3
1.3 Operating System
Security within Total EDP
Protection.............................4
1.4 An Example of a
Security Flaw........................5
2. Security Enhancements of
Operating Systems...................6
2.1 Detection Controls..........6
2.2 Corrective-Preventive
Controls................................7
a. Hardware......................7
b. Software........................8
c. User Action...................9
d. Administrative-
Physical........................10
3. Taxonomy of Integrity
Flaws........................................10
3.1 Introduction....................10
3.2 Taxonomy of Integrity
Flaws..................................10
3.3 Class of User................11
a. Application Users......11
b. Service Users............12
c. Intruder........................12
3.4 Class of Integrity
Flaw.....................................12
3.5 Class of Resource.......12
3.6 Category of Method.......13
3.7 Category of
Exploitation........................13
3.8 Detailed Description of
Operating System Security
Flaws..................................13
a. Incomplete Parameter
Validation.....................14
b. Inconsistent Parameter
Validaiton.....................16
c. Implicit Sharing of
Privileged/Confidential
Data...............................17
d. Asynchronous
Validation/Inadequate
Serializaition.................19
e. Inadequate Ident-
ification/Authorization/
Authentication...............22
f. Violable Prohibition/
Limit...............................23
g. Exploitable Logic
Error................................23
4. IBM OS/MVT............................26
4.1 Introduction....................26
4.2 Overview of OS/MVT
History.................................27
4.3 IBM/360 and OS/MVT
Prevention Concepts.......28
a. Hardware Isolation
Features.......................28
b. Control Access
Features.......................28
c. Integrity Monitoring
and Surviellance.........29
4.4 Summary........................29
4.5 Operating System
Integrity Flaws...................30
5. UNIVAC 1100 Series
Operating System...................31
5.1 Introduction....................31
5.2 Design Criteria of the
Operating System.............32
5.3 1108 Architecture..........32
a. Memory Interface.......34
b. System Control..........34
5.4 Integrity Features..........34
a. User Control...............34
b. States of Execution...35
c. Protection of
Permanent Files.........35
d. Protection of
Magnetic Tapes..........35
e. Audit Trails..................36
f. Role of System
Console Operator........36
g. Impact of System
Degradation.................37
5.5 Summary........................37
5.6 Operating System
Integrity Flaws...................38
6. Bolt Beranek and Newman
TENEX......................................39
6.1 Introduction to
TENEX................................40
6.2 Typical Use of
TENEX................................40
6.3 Overview of TENEX
Hardware Architecture
and Integrity Features......42
a. CPU.............................42
b. Virtual Memory
Hardware.....................43
c. Peripherals.................43
6.4 Operating System
Design and Integrity
Features.............................44
a. File Protection............45
b. Directory Protection...45
c. Process Protection....46
6.5 Summary........................47
6.6 Operating System
Integrity Flaws...................48
a. Existing Flaws............48
b. Flaws that have
been Fixed...................49
7. Summary and
Conclusions............................52
Glossary......................................54
Bibliography................................59
References.................................62
Keywords
BBN-TENEX, IBM OS/360, UNIVAC 1100Series OS, operating system security, software security, security flaws, taxonomy of integrity flaws
Subject
Enhancements of Computer Operating
