No d’ordre : 2008 ISAL 0117

Année 2008

Thèse

Software Security Models for Service-Oriented Programming (SOP) Platforms À présenter devant

l’Institut National des Sciences Appliquées de Lyon École Doctorale Informatique et Information pour la Société pour l’obtention du

Grade de Docteur spécialité informatique par

Pierre PARREND À soutenir le 9 Décembre 2008

Devant le jury composé de Président : Directeurs : Rapporteurs :

Didier Donsez, Stéphane Ubéda, Stéphane Frénot, Didier Donsez, Ralf Reussner,

Examinateurs : Ciaran Bryce, Pierre-Etienne Moreau,

Professeur, Professeur, Maître de conférences, Professeur, Professeur, Maître d’enseignement et de recherche, Chargé de Recherche, habilité à diriger des recherches,

Université Grenoble I INSA de Lyon INSA de Lyon Université Grenoble I Technische Universität Karlsruhe Université de Genève INRIA Lorraine

Thèse effectuée au sein du Centre d’Innovation en Télécommunications et Intégration de Services (CITI) de l’INSA de Lyon et de l’équipe Architecture Réseaux et Systèmes (Ares) de l’INRIA Rhône-Alpes

This page intentionnaly left blank.

Résumé Les plates-formes dynamiques de services (SOP, pour ‘service-oriented programming’) sont des environnements d’exécution génériques qui définissent un modèle d’architecture logicielle structuré: les composants communiquent par le biais d’interfaces explicites, ce qui facilite la configuration et l’évolution de ces systèmes. Les plates-formes SOP utilisent leur environnement réseau pour réaliser des tâches fonctionnelles, mais également pour améliorer leur capacité de gestion et d’évolution. Elles sont exploitées dans des contextes variés, des serveurs d’application aux systèmes embarqués médicaux ou automobiles. La flexibilité apportée par les plates-formes SOP permet l’intégration de composants de plusieurs sources aussi bien lors de la conception qu’à l’exécution. Cette tendance induit cependant un risque important. Peu d’outils existent pour évaluer la qualité des systèmes résultants, et aucun ne garantit que les composants sélectionnés ne sont pas malicieux. Dans des contextes applicatifs tels que les systèmes e-Business ou les systèmes embarqués sensibles, l’existence d’attaques n’est pas à exclure. L’assurance de sécurité logicielle (Software Security Assurance) définit des méthodes pour le développement d’applications sûres, mais se concentre sur les systèmes monolithiques. Son principe est le suivant: les vulnérabilités doivent être identifiées et résolues tôt dans le cycle de vie pour éviter les attaques lors de l’exécution et limiter les coûts de réparation. Cependant, cette approche ne peut s’appliquer directement aux applications à composants, où le développement n’est pas nécessairement contrôlé par l’intégrateur, et où l’intégration peut avoir lieu à l’exécution de manière automatisée. Nous proposons par conséquent de réaliser une analyse de sécurité pour une plate-forme SOP de référence, la plate-forme OSGi, et de fournir des mécanismes de protection adaptés aux besoins ainsi identifiés. L’analyse de sécurité de la plate-forme OSGi est réalisée par une méthode spécifique, SPIP , le Processus Spirale de Prévention d’Intrusion (Spiral Process for Intrusion Prevention). Elle permet l’évaluation des vulnérabilités du système cible et de la protection apportée par les mécanismes de sécurité associés. Le résultat de l’analyse est : les vulnérabilités de la plate-forme Java/OSGi, et les vulnérabilités des composants SOP Java. Plusieurs mécanismes de protection sont développés pour prévenir l’exploitation des vulnérabilités identifiées. Ils sont implémentés dans la plate-forme elle-même et au niveau des composants. OSGi Robuste (Hardened OSGi) est un ensemble de recommandations pour la mise en œuvre de plates-formes OSGi résistantes. CBAC, le contrôle d’accès basé composants (Component-based Access Control) est un mécanisme de contrôle d’accès qui vérifie lors de l’installation qu’un composant n’exécute que les appels explicitement autorisés. Son objectif est d’être plus flexible que le gestion de sécurité Java, de garantir que seuls les composants valides soient installés et de réduire autant que possible le coût de vérification en terme de performance. WCA, l’analyse de composants faibles (Weak Component Analysis), est un outil pour identifier les vulnérabilités exploitables dans les composants SOP selon l’exposition des classes: les objets partagés tels les services SOP, les classes partagées et les classes internes

v

des composants ne sont pas concernés par les mêmes vulnérabilités. Nos propositions sont validées par leur intégration avec une JVM sécurisée dédiée aux applications OSGi, la JnJVM. Les propriétés de sécurité de l’environnement ainsi réalisé sont encourageantes. Une présentation de cette thèse en français est disponible dans [PF08d].

vi

Summary Service-oriented programming (SOP) platforms are generic execution environments enforcing a proper architectural model for applications: software components communicate through well-defined interfaces, which eases the configuration and evolution of applications. These platforms take advantage of their networked environment to perform distributed functional tasks, but also to enhance their management and evolution capacity. They are involved in numerous contexts, from applications servers to embedded health-care and automotive systems. The increased flexibility brought in by SOP platforms enables to integrate components provided by different issuers during the design phase and even at runtime. This trend has nonetheless a serious drawback. Few tools exist to assess the actual quality of the resulting systems, and none is available to guarantee that the selected components do not perform malicious actions. In applications such as e-Business systems or sensitive embedded systems, the intervention of attackers can not be excluded. Software Security Assurance provides methods for the development of secure applications, but focuses on monolithic systems. Its principle is the following one: vulnerabilities should be identified and solved as early as possible in the life-cycle to avoid runtime abuses and to reduce patching costs. However, this approach is not well-suited for component applications: the development process is not controlled by the integrator. When the integration is performed at runtime, no human intervention is possible to evaluate the quality of the components. We therefore propose to perform a security analysis of one prototypical SOP platform, the OSGi platform, and to provide protection mechanisms tailored to the identified requirements. The security analysis of the OSGi platform is performed with a dedicated method we define for security benchmarking, SPIP , the Spiral Process for Intrusion Prevention. It supports the assessment of vulnerabilities of the target system and of the protective power of associated security mechanisms. The output of the analysis is: the vulnerabilities of the Java/OSGi platform, and the vulnerabilities of Java SOP components. Several protections mechanisms are developed to prevent the exploitation of identified vulnerabilities. They are implemented in the platform itself and at the component level. Hardened OSGi is a set of recommendations for building more robust implementations of the OSGi platform. CBAC, Component-based Access Control, is an access control mechanism that verifies at install time that a component only performs calls it is authorized to. It intends to be more flexible than the Java security manager, to ensure that policy-compliant components only are installed and to reduce as much as possible the verification performance overhead. WCA, Weak Component Analysis, is a tool for identifying exploitable vulnerabilities in SOP components, according to the exposition of classes: shared objects, i.e. SOP services, shared classes, and component internal classes are not plagued by the same type of vulnerabilities. Our propositions are validated through their integration with a secure JVM dedicated to OSGi applications, the JnJVM. The resulting environment proves to have very encouraging security benchmarking results.

viii

Remerciements Je remercie tout particulièrement Stéphane Frénot, mon directeur de thèse, qui a permis la réalisation de cette thèse dans l’équipe INRIA-Ares, puis INRIA-Amazones, du laboratoire CITI de l’INSA de Lyon. Il a su à la fois me fournir un environnement de recherche et une vision technique forts, et me donner l’autonomie de travailler sur des problématiques complémentaires mais différentes de celles qui existaient dans l’équipe. Merci également à Stéphane Ubéda, en tant que co-directeur de thèse et directeur de laboratoire, qui a fait preuve d’une efficacité constante dans ses deux fonctions, et qui a permit à cette thèse, comme à tant d’autres, de se dérouler dans de bonnes conditions. Je remercie Didier Donsez, Professeur à l’Université de Grenoble, et Ralf Reussner, Professeur à l’Université de Karslruhe, Allemagne, d’avoir accepté d’être les rapporteurs de ce document. Et en particulier d’avoir permis sa rédaction en anglais. Je remercie également Ciarán Bryce, de l’Université de Genève, et Pierre-Etienne Moreaux, de l’INRIA Lorraine, de s’être intéressés à mon travail et d’avoir accepté de faire partie du jury de thèse. Plusieurs personnes ont également contribué au contenu de cette thèse. Je citerai Gaël Thomas pour ses retours concernant la sécurisation de la machine virtuelle Java, Peter Kriens pour ses remarques concernant l’intégration de mes propositions dans les plates-formes OSGi, et Capers Jones pour les précieuses informations qu’il m’a fournies sur les processus de qualité logiciels. Je remercie l’équipe Middleware du CITI pour leur présence au long de ces trois années, les discussions de recherche et le soutien logistique que tous ont pu m’apporter pour la réalisation et l’organisation de la soutenance de thèse: Yvan Royon, Frédéric Le Mouël, Amira Ben Amida, Hajer Chamekh, Noha Ibrahim. Je remercie également tous ceux qui m’ont fait découvrir le monde de la recherche : Bertrand David, Professeur à l’Ecole Centrale de Lyon, qui m’a encadré pendant le Master recherche, ainsi que Isabelle Augé Blum, du CITI, avec qui j’ai réalisé mon projet de fin d’étude d’Ingénieur INSA, sans qui je n’aurais sans doute pas décidé de m’orienter dans cette voie. Un grand merci à tous les amis que j’ai rencontré pendant cette thèse, et à ceux que j’ai retrouvé après ces années de travail parfois un peu trop intensif. Je remercie enfin mes parents, qui m’ont permis de réaliser un Master recherche après la fin de mes études. Je remercie mes beaux parents d’avoir supporté mes humeurs de thésard ne sachant pas toujours ce qu’il cherche. Et je remercie naturellement Pauline, ma chère et tendre épouse, qui a sacrifié ses soirées et ses week-ends au moins autant que moi pendant cette période, et qui m’a apporté un soutien bien indispensable à l’aboutissement de cette thèse.

ix

Contents 1 Introduction 1.1 The Argument of this Thesis . . . . 1.2 Context . . . . . . . . . . . . . . . . 1.2.1 The Software Crisis . . . . . 1.2.2 SOP Platforms . . . . . . . . 1.2.3 SOP Platforms and Security 1.3 Document Outline . . . . . . . . . .

. . . . . .

3 3 4 4 5 7 9

2 Preliminary Development: secure Deployment for the OSGi Platform 2.1 Bundle Digital Signature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Bundle Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 OSGi Bundles as Java Protection Domains . . . . . . . . . . . . . . . . . . .

13 13 15 18

I

19

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

Background and Related Works

3 Software Security 3.1 What is Software Security ? . . . . . . . . . 3.1.1 Definition . . . . . . . . . . . . . . . 3.1.2 The Domains of Computer Security 3.1.3 Software Security Assurance . . . . . 3.2 Security Requirement Elicitation . . . . . . 3.2.1 Vulnerability Identification . . . . . 3.2.2 Software Security Benchmarking . . 3.3 Techniques for Software Security . . . . . . 3.3.1 Secure Architecture and Design . . . 3.3.2 Secure Coding . . . . . . . . . . . . 4 Security for Java Platforms 4.1 The default Security Model . . . 4.1.1 The Java Language . . . . 4.1.2 Bytecode Validation . . . 4.1.3 Modularity . . . . . . . . 4.2 Vulnerabilities . . . . . . . . . . . 4.2.1 Vulnerabilities in the Java 4.2.2 Vulnerabilities in Code . . 4.3 Security Extensions . . . . . . . . 4.3.1 Platform Extensions . . . 4.3.2 Static Analysis . . . . . . 4.3.3 Behavior Injection . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Virtual Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

21 21 21 23 24 26 26 29 32 33 34

. . . . . . . . . . .

39 40 40 41 42 45 45 46 50 50 52 54

xi

Contents 5 Service Oriented Programming (SOP) Platforms and Security 5.1 Service-oriented Programming . . . . . . . . . . . . . . . . . 5.1.1 What is SOP ? . . . . . . . . . . . . . . . . . . . . . 5.1.2 Service Dependency Resolution . . . . . . . . . . . . 5.1.3 Contribution to Security . . . . . . . . . . . . . . . . 5.2 Existing Java SOP Platforms . . . . . . . . . . . . . . . . . 5.2.1 Structure . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . 5.2.3 Security . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 The OSGi Platform . . . . . . . . . . . . . . . . . . . . . . 5.3.1 Principles . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2 Service Management . . . . . . . . . . . . . . . . . . 5.3.3 Security . . . . . . . . . . . . . . . . . . . . . . . . .

II

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . .

Security Analysis

6 A Methodology for Security Analysis of Execution Environments 6.1 Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.1 Security Analysis for Execution Environments: a Terra Incognita of Software Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.2 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.3 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 SPIP - Spiral Process for Intrusion Prevention . . . . . . . . . . . . . . . . . 6.2.1 Analysis Course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.2 An Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.3 Security Benchmarking . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Using SPIP for SOP platforms . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.1 Process Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

57 57 57 58 59 60 60 61 63 65 65 67 68

73 75 75 76 76 77 78 79 80 82 84 84 86

7 Executing untrusted Java/OSGi Bundles: A Vulnerability Assessment 87 7.1 Defining Tools for Security Analysis . . . . . . . . . . . . . . . . . . . . . . . 88 7.1.1 Taxonomies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 7.1.2 The descriptive Vulnerability Pattern for Component Platforms . . . . 89 7.1.3 The Protection Rate Metric . . . . . . . . . . . . . . . . . . . . . . . . 90 7.2 Vulnerability Assessment of Java/OSGi Systems and Applications . . . . . . 92 7.2.1 Vulnerabilities in the Java/OSGi Platform . . . . . . . . . . . . . . . . 92 7.2.2 Vulnerabilities in Java/OSGi Bundles . . . . . . . . . . . . . . . . . . 101 7.3 Protection Assessment of Java/OSGi Systems and Applications . . . . . . . . 108 7.3.1 Protection Assessment of standard Implementations of the OSGi platform110 7.3.2 Protection Assessment of Java/OSGi Bundles . . . . . . . . . . . . . . 110 7.3.3 Evaluation of Java Permissions . . . . . . . . . . . . . . . . . . . . . . 111

III Hardening the Java/OSGi SOP Platform

xii

113

Contents 8 Mechanisms for secure Execution in the Java/OSGi Platform 8.1 Hardened OSGi . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.1 Recommendations . . . . . . . . . . . . . . . . . . . 8.1.2 Implementation . . . . . . . . . . . . . . . . . . . . . 8.1.3 Results of Security Benchmarking . . . . . . . . . . 8.2 The Install Time Firewall Security Pattern . . . . . . . . . 8.3 Component-based Access Control - CBAC . . . . . . . . . . 8.3.1 Requirements . . . . . . . . . . . . . . . . . . . . . . 8.3.2 Definition of CBAC . . . . . . . . . . . . . . . . . . 8.3.3 Implementation . . . . . . . . . . . . . . . . . . . . . 8.3.4 Results of Security Benchmarking . . . . . . . . . . 8.4 Weak Component Analysis - WCA . . . . . . . . . . . . . . 8.4.1 Requirements . . . . . . . . . . . . . . . . . . . . . . 8.4.2 Definition of WCA . . . . . . . . . . . . . . . . . . . 8.4.3 Implementation . . . . . . . . . . . . . . . . . . . . . 8.4.4 Results of Security Benchmarking . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

. . . . . . . . . . . . . . .

115 116 116 118 119 120 124 124 125 126 130 133 133 133 135 139

9 An integrated secure Java/OSGi Execution Environment 9.1 Hardened OSGi over a secure Java Virtual Machine . . 9.1.1 Resource Isolation for OSGi Platforms . . . . . . 9.1.2 Implementation . . . . . . . . . . . . . . . . . . . 9.1.3 Results of Security Benchmarking . . . . . . . . 9.2 Developing secure Java/OSGi Bundles . . . . . . . . . . 9.2.1 Identified Constraints . . . . . . . . . . . . . . . 9.2.2 Security in the Software Development Life-Cycle 9.3 Security Benchmarking of the integrated System . . . . 9.3.1 Protection Mechanisms for Java Systems . . . . 9.3.2 Protection against Vulnerabilities . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

143 143 143 145 145 147 148 149 150 150 150

. . . . . . . . . .

. . . . . . . . . .

IV Conclusions and Perspectives

157

10 Conclusions and Perspectives 10.1 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1.1 Methodology for Security Analysis . . . . . . . . . . . . . . . . . 10.1.2 A secure Execution Environment . . . . . . . . . . . . . . . . . . 10.1.3 Secure Components . . . . . . . . . . . . . . . . . . . . . . . . . 10.1.4 Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2 Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.1 Open Requirements for building Secure Execution Environments 10.2.2 Open Requirements for Industrial Use Cases . . . . . . . . . . . 10.3 Conclusions on the Thesis’ Argument . . . . . . . . . . . . . . . . . . . .

159 159 159 160 160 161 162 162 163 166

V

Appendix

A Definitions

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

167 169

xiii

Contents B Vulnerability Catalog Entries 175 B.1 The Malicious Bundle Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . 175 B.2 The Vulnerable Bundle Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . 178 C Listings 181 C.1 Examples of Attacks against the Java/OSGi Platform . . . . . . . . . . . . . 181

xiv

List of Figures 1.1 1.2 2.1 2.2

Average of defects found throughout the life-cycle of software and the cost to repair them . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The weaknesses in a SOP dynamic application . . . . . . . . . . . . . . . . .

6 8

2.3 2.4

Performances of bundle digital signature validation, according to the class number Performances of bundle digital signature validation, according to the class number, with repetitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Life-cycle of OSGi bundles with security management support . . . . . . . . . Overview of preliminary developments . . . . . . . . . . . . . . . . . . . . . .

16 17 18

3.1 3.2

The security domains of computer security in the applicative stack . . . . . . The life-cycle phases of software security assurance . . . . . . . . . . . . . . .

23 25

4.1 4.2 4.3 4.4

Architecture of a Java Virtual Machine . The process of Bytecode verification . . The hierarchy of class loaders . . . . . . Existing protection mechanisms for Java

. . . . . . . . . . . . . . . systems

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

39 42 43 55

5.1 5.2 5.3 5.4

The timeline for the Inversion of Control pattern The structure of a SOP platform . . . . . . . . . OSGi platform model . . . . . . . . . . . . . . . OSGi component model . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

59 60 65 66

6.1 6.2

Overview of SPIP - Spiral Process for Intrusion Prevention . . . . . . . . . . Stepwise securization of a system: the cumulated effect of complementary protection mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

79

7.1 7.2 7.3

15

83

7.6 7.7

Relative importance of intrusion techniques in the Java/OSGi platform . . . 99 Taxonomy for consequence type in the Java/OSGi platform . . . . . . . . . . 100 An example scenario for an attack against a synchronized method: sequence diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 An example of a service with vulnerable method: sequence diagram . . . . . 103 An example of a service with vulnerable method that is abused through malicious inversion of control: sequence diagram . . . . . . . . . . . . . . . . . . . 104 Taxonomy: vulnerability categories in the Java/OSGi platform . . . . . . . . 109 Performance of Felix OSGi platform without and with the Java security manager112

8.1 8.2 8.3 8.4

Performances of Hardened Felix . . . . . . . . . . . . . . . . . . . . . . . . The Install Time Firewall security pattern: sequence diagram . . . . . . . The Component-based Access Control security pattern: sequence diagram Performances of OSGi security: CBAC check only . . . . . . . . . . . . .

7.4 7.5

. . . .

. . . .

118 123 125 129

xv

List of Figures

xvi

8.5 8.6 8.7 8.8 8.9

Performances of OSGi security: Digital signature and CBAC checks . . . . . The structure of the WCA Security Pattern . . . . . . . . . . . . . . . . . . . Performances of WCA check for sample vulnerable bundles . . . . . . . . . . Performances of WCA check for Felix trunk bundles . . . . . . . . . . . . . . Performances of WCA check for Felix trunk bundles, according to bundle size

129 134 138 139 140

9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8

Hardened OSGi with CBAC over the JnJVM Virtual Machine . . . . The publication process for Hardened OSGi applications . . . . . . . . Existing and proposed protection mechanisms for Java systems . . . . The type of vulnerabilities in a SOP platform . . . . . . . . . . . . . . Security benchmark for a default OSGi platform . . . . . . . . . . . . Security benchmark for Hardened OSGi . . . . . . . . . . . . . . . . . Security benchmark for Hardened OSGi over JnJVM . . . . . . . . . . Taxonomy: vulnerability categories and related security mechanisms Java/OSGi platform . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . in the . . . .

145 150 151 152 153 154 154

10.1 Overview of development and scientific contributions . . . . . . . . . . . . . .

161

155

List of Tables 2.1

Behavior of several tools and frameworks in the presence of unvalid archives .

7.1

Example of the descriptive vulnerability pattern: Management utility freezing - infinite loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Taxonomy: implementations of the vulnerabilities in the Java/OSGi platform Taxonomy: implementations of the component vulnerabilities in the Java/OSGi platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Taxonomy: consequences of the attacks that exploit vulnerabilities in Java/OSGi component interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Protection rate for mainstream OSGi platforms . . . . . . . . . . . . . . . . . Protection rate for the Java security manager . . . . . . . . . . . . . . . . . .

7.2 7.3 7.4 7.5 7.6 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 9.1 9.2

Protection rate for the Hardened OSGi platform . . . . . . . . . . . . . . . . The Install Time Firewall security pattern . . . . . . . . . . . . . . . . . . . Example of a CBAC policy file . . . . . . . . . . . . . . . . . . . . . . . . . . Protection rate for the CBAC protection mechanism, and comparison with the Java Permissions alternative . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example of the formal Vulnerability Pattern: Synchronized Method Call . . . Example of the Policy for Reaction to Vulnerability . . . . . . . . . . . . . . . Example of a WCA Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . Protection rate for the WCA and complementary protection mechanisms . . .

14 91 95 106 107 110 112 120 122 128 131 135 136 139 141

Protection rate for the OSGi platform over JnJVM for platform vulnerabilities 146 Protection rate for Hardened OSGi with CBAC and WCA over JnJVM for platform and component vulnerabilities . . . . . . . . . . . . . . . . . . . . . 147

10.1 Developed Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

162

xvii

Listings 4.1 4.2 4.3

Example of a counterintuitive infinite loop in the Java language . . . . . . . . Private inner class and methods: source code . . . . . . . . . . . . . . . . . . Private inner class and methods: Bytecode . . . . . . . . . . . . . . . . . . .

7.1

Example of malicious code in an OSGi bundle: infinite loop in bundle activator 93

C.1 Example of malicious code in OSGi bundle: recursive thread creation . . . . . C.2 Bundle that launches a bundle that is hidden inside it . . . . . . . . . . . . . C.3 Example of malicious code in OSGi bundle: memory load injection . . . . . .

46 47 48

181 181 182

xix

Terms are used in a manner that aims at being coherent, and at providing a comprehensive understanding of the subject of study. When no consensus arises in the community, they can be used with slightly different meanings than in the work of other authors. Terms that are defined in Appendix A are marked with a star (*) at their first occurrence in each chapter.

Introduction

1

1.1 The Argument of this Thesis Dynamic applications are applications that can evolve at runtime, i.e. can be reconfigured and extended without the need for rebooting their execution environments*. They can thus gain new functionalities and adapt themselves to their unstable environment. Dynamic applications exploit the component-based software engineering (CBSE) [SGM02] and service-oriented programming* (SOP) [BC01] paradigms to enhance development agility and runtime flexibility. They are often built on top of virtual machines and take advantage of the portability they provide and of the robustness properties of high level languages. These paradigms are integrated in service-oriented programming platforms* which provide component* binding through local services*1 . Through advanced features such as the discovery of components in the environment, they pave the way for groundbreaking applications in the domains of connected homes, pervasive environments, or application servers. Dynamic applications can be built according to three trust models: • Legacy applications, where all components are developed by the same provider. • Design time integration of applications, where components are developed by various providers and integrated by the application architect. In this case, the architects trust the component providers and can test the code to challenge this trust. • Automated runtime integration in ‘open dynamic applications’. In this case, the components are discovered dynamically in the environment by providers on which little to no control exists. The only guarantee that is usually set is the presence of a digital signature* which certifies that the code is provided by a known entity*. Consequently, no functional or quality evaluation is performed. The last configuration would enable to build truly dynamic applications which evolve according to their environments. However, this vision would require that guarantees can be set on the component behavior related to component functionalities, quality, or benevolence. Only specific techniques such as proof-carrying-code [Nec97] for algorithmic safety exist so far. They only enable to express specific properties such as the absence of infinite loops or of memory leaks. No comprehensive method exists to manage the security* issues generated by open dynamic applications. A recent trend in application security* consists in creating software that is provably builtin secure, i.e. on which assertions related to the code property can be expressed. It is what 1

Service-oriented Programming (SOP) provides therefore a perspective on software components. It is not to be confused with Service-oriented architectures (SOA), which enable to interact remotely, although both can be used complementarily

3

1 Introduction is called ‘Software Security’* [McG06]. Its main objectives is to avoid the presence of vulnerabilities* in the architecture and in the code of applications as early as possible in the life-cycle. We consider that the application of such an approach can help increase the security status of dynamic applications, i.e. their ability to withstand attacks*, provided their specific properties are taken into account. Consequently, the work of this thesis is motivated by the following methodological assumption: Secure SOP platforms are required to make the vision of dynamic applications a reality. This challenge is made realistic thanks to the intrinsic properties of available SOP platforms but require to enforce additional, new security paradigms such as ‘software security’. The experiments that are reported in this thesis are conducted on the Java/OSGi platform. A common claim in the OSGi community is that this execution environment is a very secure one since it supports a scheme for component isolation that is more complete than many others. It is even sometimes considered as the universal local middleware, because it provides facilities for component management that are exploited by several other SOP platforms or Web servers. It is actually designed with promising properties: component isolation through class loaders which enables to load each component is a specific naming space, important set of execution permissions and specific security mechanisms that are pervasive to the specifications. However, very few use cases are reported that would confirm that the OSGi platform is suited to withstand adversarial environments. Moreover, very few tools are broadly available to support the secure execution of OSGi systems*. The choice of the target system is therefore motivated by the following technical assumption: The Java/OSGi platform is a promising but currently immature SOP platform for building secure systems for dynamic applications. The implementation of protection mechanisms for the OSGi platform must take its specific properties into account: • Components can be discovered dynamically from the environment, from providers that are not necessarily trusted. • Security checks can thus only be mandatorily performed when the components are available, i.e. when they are downloaded on the platform. They can rely on component Bytecode only.

1.2 Context 1.2.1 The Software Crisis Middleware development and software engineering are structured around the response they provide to the successive crises of software development. The historical crisis is on the way to be resolved by manageable software such as SOP platforms. As the number of networked

4

1.2 Context economically sensitive applications increase dramatically, a second crisis is gaining attention: the software security crisis. The ‘Ball of Mud’ and the emergence of manageable software The historical ‘software crisis’ is due to the important complexity of software programs related to the available development methodologies. The term dates back to the ‘Software Engineering’ NATO Conference of 1968 [Ran96], where the requirements for a proper management of software development processes have first been explicited. The dramatic increase of hardware speed made it hard for developers to build programs that could exploit the available resources while controlling the actual duration of the software projects. The term has been used until the 1990’s, where the web forced the number of programmers to explode. The principles of the requirements for managing the crisis are known since Dijskra’s Turing Award lecture, The Humble Programmer [Dij72]: systems have to be intellectually manageable; they should be organized hierarchically; it should be possible to extract proof of correctness. These requirements represent two kinds of constraints: they can be either embedded in the programming language, or may require external theorem provers. Over the year, languages and systems have evolved to avoid the creation of ‘Big Balls of Mud’ [FY97], i.e. monolithic software with deeply intertwined code. Techniques have been developed to enforce better programming models: Object-oriented programming, componentbased development. SOP platforms are currently the most complete solution. They rely on efficient execution platforms and enforce programming best practices. They enable to achieve both better productivity and better code quality. The software security crisis and software security assurance * As the management of software projects seems to be a concern of less intensity, another crisis is gaining attention: the software security crisis. Since more and more applications are developed for connected devices, the weaknesses that once where simple bugs turn out to be vulnerabilities that enable attacker to abuse the systems. The response of the software engineering community to the software security crisis is called software security assurance. It consists in integrating security measures throughout the development life-cycle to prevent the presence of as much vulnerabilities as possible in the code. Of course, the objective is to limit the cost caused by security failures. Figure 1.1 [Jon99] shows the average of defects found throughout the life-cycle of software and the cost to repair them2 . One error that costs 1 US dollar ($) to repair during the design phase costs 25 $ during coding, between 100 and 150 $ during tests and up to 16 000 $ if it is corrected post release. This estimation does not take into account the by-products of such errors such as reputation loss. Strong efforts are therefore advocated to enforce software security early in the life-cycle.

1.2.2 SOP Platforms SOP platforms are characterized by the programming paradigms they rely on: Virtualization*, modularity* and service-oriented programming. The target system of our analysis, the OSGi platform, is an example of a widespread SOP platform. 2

The graph is reproduced with the kind authorization of the author

5

1 Introduction

Figure 1.1: Average of defects found throughout the life-cycle of software and the cost to repair them Virtualization and Modularity Virtualization is a broad term that refers to the abstraction of computer resources. At the application level, it enables to control the execution of full programs while being independent of both the underlying hardware and operating system. It is suited for resource constraint environments: it does not emulate a full computer but only provides required support for executing general purpose applications. Modularity is a software design technique that increases the extent to which software is composed from separate parts called modules or components. It aims in particular at making these modules re-usable [McI69]. Components can serve different goals: execution unit [Hal01], service unit or deployment unit. These goals can co-exist in a single component model. The type of components that we are concerned about is Off-the-Shelf components* (OTS). OTS is a generic term that appoints readily available components, as opposed to components that are built according to pre-defined constraints. OTS can be commercial OTS (COTS) [CL00, Voa98], open source components, legacy software or components that are discovered at runtime in the environment through repositories such as the OSGi Open Bundle Repository, OBR [AH06]. Service-oriented programming Service-oriented programming [BC01] consists in letting software components installed on the same execution platform communicate through local services. It is presented in Chapter 5. It is built on Object-Oriented-Programming (OOP) and component models, and emphasizes the possibility of improved software re-use [SVS02] and the evolution of applications at runtime. The OSGi platform One of the platforms of choice for executing dynamic Java applications is the OSGi platform [All07a]. It has originally been designed for lightweight and connected devices, for instance automotive entertainment systems or health care systems. Its clean deployment and programming model fosters its adoption in numerous other contexts such as

6

1.2 Context application servers. Components can be discovered, installed and managed throughout their life-cycle and can be un-installed to let place to other ones. Moreover the memory footprint of the platform itself is quite reduced. These mechanisms prove to be useful for a wide range of Java-based systems. More and more application servers such as JBoss, Jonas, Weblogic, and Integrated Development Environments (IDEs) such as Eclipse use it to manage components and their dependencies. However, and even though the standard Java security model has been tailored for OSGi, few work related to the new security challenges brought in by this platform have been released.

1.2.3 SOP Platforms and Security SOP platforms exploit the security features of their building blocks, such as the virtual machine and the modular support. Java SOP platforms, of course, heavily rely on the Java security model. However, the highly connected and extensible applications they enable to build introduce new security challenges. The main attack vectors* are presented, as well as the limitations of mainstream software security assurance approaches. Virtualization and modularity Virtualization provides a layer that isolates the applications from the underlying system. The benefits are manyfold. First, isolation from the underlying operating system enhances the security of the system. Code can no longer access the OS. Only the VM which is smaller and contains well-defined interfaces must be secured. Next, high level languages are often safer as native languages because they support advanced security models and are designed with built-in properties such as type-safety. Moreover, virtualization enables the reuse of secure architectures and thus prevents the multiplication of ad-hoc less tested solutions. Modularity can be considered in itself as another good practice for building secure systems [MTS04]. Building an application out of small components with strict boundaries between them enhances the isolation between its different parts and makes the system far more testable as a monolithic structure. This increases the stability of the system and makes it less exposed to denial-of-service attacks or ill-defined behavior. Moreover, component models often integrate additional patterns that enhance the security level* of systems that are built on top of them [MTS04] even though they are not explicitly meant to be protection mechanisms. Such patterns are information hiding, abstraction, principle of least authority, absence of global namespace, use of patterns of safe cooperation, to only cite a few. The Java security model The security in Java SOP applications in based on the Java security model. It is composed by two complementary elements: the default features of the Java language and virtual machine, and the Java security manager. The default features of the Java language and VM are designed to avoid abuses of the system by malicious programs (See Chapter 4). They are type safety, Bytecode validation at class loading, automated memory management through garbage collection, and modularity. Modularity is enforced by the loading of classes from various origins in different class loaders to prevent undue access and name conflicts. The Java security manager enforces access control in Java applications. It is based on the notions of ‘principal’*, i.e. the owner of a code component, and a ‘domain’, i.e. the set of rights a principal has for executing sensitive code. Its principle is the following: in dangerous

7

1 Introduction methods, calls to the security manager enable to check the availability of sufficient execution rights for all principals of the current call stack. The Java security model is designed to enable the safe execution of untrusted Applets and in particular to prevent Bytecode forgery. Its support for secure modularity is limited to a comprehensive but not flexible access control mechanism. One of the goals of our work is to determine to what extend it is suitable for dynamic applications, and whether it complies with their specific requirements. Attack vectors Introducing flexibility in applications also means introducing new ways for malicious actors to exploit the system. For typical applications of SOP platforms such as web applications, multimedia automotive or health systems this is clearly not acceptable and prevents vendors from exploiting the full potential of the platform. We strongly believe that the dynamism of SOP platforms can only be leveraged if its security implications are well understood and risks of malicious exploitation reduced. At the moment we began this work this was clearly not the case, in particular in the context of the OSGi platform. Our objective is therefore to identify the actual threats to SOP platforms and potential solutions that would enable users to take the best out of this technology. Figure 1.2 shows an overview of the threats on a SOP system and the subject of this thesis: exploiting weaknesses of SOP platforms.

Figure 1.2: The weaknesses in a SOP dynamic application The following attack vectors can be exploited. • Bundle* deployment: the code can be intercepted, read and/or modified. • Application interfaces: Graphical User Interfaces (GUI) and remote interfaces such as RMI or Web Services which are often used in conjunction with the platform. • Attack against the JVM from the host system. • Execution of malicious bundles inside the platform. We choose to focus to the attack vectors specific to SOP platforms. The deployment essentially induce development effort. Execution of malicious bundles needs to be analyzed thoroughly to identify exploitable vulnerabilities it introduces and to propose new protection mechanisms Limitations of software security assurance approach The ‘software security assurance’ approach is relatively straightforward for monolithic systems, where the code is developed

8

1.3 Document Outline by a single organization. For dynamic applications, where the code originates from third party providers, it highlights a paradox: code should be ‘built-in’ free of vulnerability, but is not necessarily available. Building secure dynamic applications therefore imply to tackle this paradox.

1.3 Document Outline Defining a secure SOP execution environment involve two complementary challenges: a methodological challenge, and a technical challenge. The methodological challenge is the relevance of the Software Security approach to build secure component platforms*. The technical challenge is the suitability of the security level service-oriented programming platforms such as the OSGi platform can provide for its use cases. The study of these challenges is performed through three main steps: a state of the art survey on these topics, the security analysis of the target system and the proposition of a set of security mechanisms that intend to solve the identified vulnerabilities. Preliminary development A pre-requisite for building secure Java SOP platforms is to ensure that the deployment process is itself secure, i.e. that components can not be modified between the time they are released and the time they are installed on a client* platform as presented in Chapter 2. No open source tools exist for the OSGi platform. We therefore developed tools to support verifying the digital signature of OSGi bundles which is more strict than the signature for standard Java Archives and a tool for signing and publishing bundles. The standard security mechanism for secure execution in Java, the enforcement of Permissions through a security manager, also need to be supported. This enables the evaluation of the security level that can be provided by an OSGi platform and provides a reference implementation for comparison with our own propositions. Part I presents a survey of the state of the art of research and technology for software security in Chapter 3, the security mechanisms in the Java world in Chapter 4 and the target system of our study, SOP platforms, in Chapter 5. The properties of the Software Security approach and related techniques are given in Chapter 3 page 21. This research and technical domain is defined. It concerns specific security aspects that are bound with the properties of the code of the application, rather than networklevel, system-level security or security issues that are tackled after the development of applications. The concept of Software Security Assurance, i.e. the process of ensuring that software is free of vulnerabilities with a certain level of confidence, is introduced. Techniques that can enforce security at the software level are of three main types independent of the considered programming language. The first approach relies on software engineering, i.e. modification of the development life-cycle and the second one on code engineering, i.e. constraints and modifications on the code. The security features of Java platforms are presented in Chapter 4 page 39. Several vulnerabilities are well-known and others are disclosed in very recent publications. The default security model for Java platforms is based on Permissions. It is also named ‘Stack-based Access Control’ since it implies the verification of the whole call stack when security checks are performed. This ensures that a method can be executed if all the callers are allowed to

9

1 Introduction access it. Various extensions to this security model are proposed. They imply either the modification of the platform, the addition of constraints on the code to be executed or behavior injection through code transformation. SOP platforms are discussed in Chapter 5 page 57. The SOP paradigm is defined. The structure of significant SOP platforms is identified and examples are given. The specific security requirements of this execution environment are identified. Lastly, the OSGi platform and its support for service-oriented programming are presented. Security aspects are discussed. Part I introduces state of the art methodologies for security analysis, and presents existing protection principles and mechanisms for Java environments. It defines SOP platforms, the target system of our analysis. Part II presents the security analysis of the target system. A dedicated methodology, SPIP , is defined in Chapter 6. The results of its application on Java SOP platforms, at the example of the OSGi platforms, are given in Chapter 7. The methodology of this thesis is presented in Chapter 6 page 75. The scientific and technical motivation for this work lies in the absence of efficient tools that enable the realization of secure systems that are based on Java SOP platforms but also on the lack of information related to the vulnerabilities that appear in such execution environments. A specific process for security analysis of complex systems, in particular execution environments, is defined. The SPIP process is a Spiral Process for Intrusion Prevention. It consists in a series of iterations of system security* evaluation and enhancement. Each iteration is built up by a Vulnerability Assessment* phase where weaknesses are analyzed and a Protection Assessment* phase where protection mechanisms are evaluated. The application of SPIP to Java SOP platforms builds the remaining of the thesis. The results of vulnerability assessment for the Java/OSGi SOP platform are presented in Chapter 7 page 87. The first step is to refine the tools for security analysis to make them fine-tuned for our target system: taxonomies and a descriptive Vulnerability Pattern* for security benchmarking* of component platforms are defined. A more generic metric, the Protection Rate, is introduced to quantify the protection level that is brought by each potential security mechanism. It can also be used to compare various implementations of the Java/OSGi SOP platform. The second step is to perform the actual vulnerability assessment. This is performed for the vulnerabilities that are induced by the Java/OSGi SOP platform itself and for the vulnerabilities that originate in the component code. Part III presents a set of solutions to the security requirements that are identified in previous part. The individual mechanisms we propose are presented in Chapter 8. An integrated secure Java/OSGi execution environment is presented in Chapter 9, and a global security assessment is performed. The mechanisms for secure execution in the Java/OSGi SOP platform that build the core of our proposition are presented and assessed in Chapter 8 page 115. An example scenario is given to highlight the requirements and benefits of the solutions. The first of these security solutions aims at solving vulnerabilities that originate in the implementation of the OSGi framework itself. It consists in a set of implementation recommendations. The next propositions are implementations of the Install Time Firewall Security Pattern*. It consists in performing security verification on the component Bytecode immediately before its installation. This approach has both advantages not to require the co-operation of the code

10

1.3 Document Outline issuer which may not be benevolent and to avoid runtime overhead. It is already used for approaches such as Proof-Carrying-Code. The first example of the Install Time Firewall is the Component-Based Access Control, CBAC. Its objective is to replace Java Permissions which prove to have several serious drawbacks with execution permissions that are computed through static analysis. The second example is the Weak Component Analysis, WCA. Its objective is to parse the code of bundles to ensure that the classes that are shared with others are free from vulnerabilities. It is also based on static analysis. Vulnerabilities are defined through a formal Vulnerability Pattern for Java classes which is distinct from the descriptive Vulnerability Pattern that is introduced for documentation purpose. Each of these mechanisms is discussed and the protection it provides is quantified. The result of the integration of these propositions with a secure JVM which enforces resource isolation between the class loaders and thus the OSGi bundles is presented in Chapter 9 page 143. The secure JVM that is used is the JnJVM which takes advantages of OSGi class loaders to create Isolates and transparently performs resource and access control. The constraints that need to be guaranteed on Java/OSGi bundles to enforce the maximal security are summarized. The security benchmarking of the integrated system validates the proposition. It highlights requirements that still need to be addressed. Part IV, Conclusions The results of this work are discussed: security benchmarking processes and tools, taxonomies and security mechanisms that are proposed greatly improve the security status of Java SOP platforms. They pave the way to further research efforts that are required to turn this set of complementary security mechanisms into a prototype that could be exploited in production environments. The state of development is also evoked. So as to ease the exploitation of this work the requirements identified in term of support for industrial use cases as well as in term of research effort are summarized. As a conclusion, the assumption that lay the main motivation of this thesis, which is presented at the beginning of this first Section, is revisited to take the lessons of the present thesis into account.

11

1 Introduction

12

Preliminary Development: secure Deployment for the OSGi Platform

2

The objective of this research work is to improve the state of the art of security* mechanisms for the Java SOP platforms* at the example of the OSGi platform. This requires that standard tools are available so as to identify their limitations, possible extensions, as well as relevant security requirements. In the case of the OSGi platform the publication of security tools is quite limited. This is first due to the fact that industry developments are often kept as added value closed source projects. This is also due to the fact that open source OSGi implementations project mostly focus on functional rather than non functional features. The development of a secure OSGi ecosystem involves to tackle the two aspects of security for extensible Java SOP platforms: secure deployment and secure execution. Preliminary development for secure deployment involves an OSGi-compliant library for verification of the digital signature* as well as a graphical tool for signing and publishing components*. Preliminary development for secure execution entails the support of bundle*-specific Java permissions.

2.1 Bundle Digital Signature Digital Signature ensures that the integrity of the bundles is preserved during the deployment process and that the issuers that sign them are authenticated and trusted. We released an open source implementation in the INRIA Gforge, under the name SFelix1 . This project is an extension of the Apache Felix implementation of the OSGi platform with the subset of the OSGi R.4 Security Layer in charge of the digital signature verification [All05a]. It is presented in [FD06], [PF06a] and [PF07b]. Implementation details are given in [PF06b]. The structure of a signed bundle and the verification algorithm are first presented. Next the criteria of validity of a digital signature are discussed for various tools and environments: the Sun jarsigner [Sun03], Java programs with a security manager [GED03], the Felix open source OSGi implementation and the SFelix extension of Felix. Structure of a signed Bundle Being a Java Archive, an OSGi bundle contains its payload, classes and resources and meta-data in the META-INF/ directory. Integrity is guaranteed by storing the value of the hash digest value for each file of the archive in the META-INF/MANIFEST. MF file. Since the Java 1.5 version all meta-data except the manifest itself are included. So as to enable the co-existence of several signers an intermediate file, the Signature file, is created and stored in the META-INF/ directory. It contains the hash 1

http://sfelix.gforge.inria.fr/

13

2 Preliminary Development: secure Deployment for the OSGi Platform values of the sections of the MANIFEST.MF file: the ‘Main-Attributes’ and each other manifest entries such as the hash value for each archive file. The digital signature in the cryptographic sense [Sch96] is extracted from this Signature file and stored in the Signature Block file together with the public key certificate(s) of the signer. The format of this Signature Block file is PKCS #7 [BK98], or CMS (Cryptography Message Syntax) [Hou04]. Digital signature uses pairs of hash and public key cryptography algorithms such as SHA-1 and DSA. Validation Criteria Whereas signing tools are available, tools for verifying the validity of digital signature are not compliant with OSGi R4 specification. This is due to the fact that OSGi requirements for digital signature are stronger than those of standard Java Archives [Sun03], where classes can be added and removed without invalidating the signature. The objective of this restriction is to prevent runtime security exceptions when classes that are not signed would be called as well as the additional overhead that is implied by related verifications at class loading. Table 2.1 shows the criteria of validity of the digital signature for various environments. Error Type

Sun Jarsigner

Java with Security Manager A

Unsigned W Archive Unknown A A Signer Addition of A A Resource Removal of A A Resource Modification R R of Resource Unvalid Order A A of Resources Signature of R R Embedded Archive Unvalid Time Of Check Test Exec A: Accept; R: Reject; W: Warning.

Felix

SFelix

R

R

R

R

A

R

A

R

W

R

A

R

W

R

Exec

Install

Table 2.1: Behavior of several tools and frameworks in the presence of unvalid archives

Sun jarsigner tool considers the modification of archive resources and invalid signature of embedded archive as errors. Unsigned archive cause a warning to be emitted. This behavior has direct implications on the execution of Java application with a security manager: errors cause the interruption of the execution but warning are not considered. For instance an unsigned archive is executed as an archive with a valid signature. Removal of signature information is thus sufficient to by-pass the default verification mechanism. Open source

14

2.2 Bundle Publication OSGi implementations does not fully solve this limitation. For instance in Felix the validity of certificates is checked but the validity of digital signature itself is handled over to the security manager. Consequently addition, removal or dis-ordering of resources are not considered as errors, which violates OSGi specification. Our SFelix implementation is developed to comply with these specifications and reject the bundles at install time whenever the digital signature is not compliant with OSGi R4 specification. Install time validation has the advantage not to impair the runtime performance of applications. Performances Figure 2.1 shows the performances of the bundle digital signature validation process, according to the class number. The tests are performed with a Sun JVM 1.5.

Figure 2.1: Performances of bundle digital signature validation, according to the class number Figure 2.2 shows the performances of the bundle digital signature validation process, according to the class number when it is repeated. The tests are performed with a Sun JVM 1.5. This mechanism will be re-used for further verifications such as Component Based Access Control (CBAC) (see Section 8.3) and Weak Component Analysis (WCA) (see Section 8.4) to provide the identity of the component provider.

2.2 Bundle Publication Principles The SF-Jarsigner tool2 supports the secure publication of OSGi bundles. It provides features to sign and publish bundles on a public repository: the Open Bundle Repository (OBR) [AH06]. OSGi client* platforms can then discover which bundles are available for installation, download them and let the SFelix verification take place. The SF-Jarsigner tool is presented in [PF07b]. Implementation details are given in [PF06b]. The functionalities of 2

http://sf-jarsigner.gforge.inria.fr/

15

2 Preliminary Development: secure Deployment for the OSGi Platform

Figure 2.2: Performances of bundle digital signature validation, according to the class number, with repetitions the SF-Jarsigner tool are first presented. Next, an extension of the OSGi bundle life-cycle is defined to enable remote management for instance through the MOSGi console3 [RF07]. The SF-JarSigner Tool contains four panels: the key management, signature and verification, OBR management and publication panels. The key management panel enables to load a pair of public/private keys pair for the digital signature from a Java keystore (JKS format). The signature and verification panel enables to select set of bundles to be signed and to specify the local directory where they are stored before being published. Alternatively it can be used to verify the validity of existing digital signature with regard to the keystore loaded in previous step. The OBR management panel enables to store information related to a set of remote file servers: access protocol, login information, archive directory, meta-data file reference. These meta-data contained in the OBR file provide data related to the bundles such as size, dependencies and various properties to enable the OSGi platform to perform dependency resolution and to load bundles only if they are required and if all dependencies are available for proper execution. New file servers can be added, removed and stored for further use. Only the FTP protocol is supported so far. The publication panel enables to upload signed bundles onto a file server. Bundles can be selected individually or sent all together. Once uploaded these bundles can be identified through the OBR file by all client platforms that maintain a reference to it. Since the bundle reference is given as a URL they do not need to be stored on the same server. The HTTP protocol is usually used to retrieve both OBR file and bundles. 3

http://mosgi.gforge.inria.fr/

16

2.2 Bundle Publication Extension of the Bundle Life-Cycle The management of OSGi platforms requires to track the state of the bundles that are installed. For instance, the MOSGi platform gives information about the bundle state, whether they are installed, active, or resolved. These states are defined in the OSGi R4 specification. However, the case of bundle rejection because of invalid signature is not considered. In this case no information about invalid bundles are available. We therefore propose to introduce an additional state in the life-cycle of OSGi bundles: the REJECTED state to enable the observation of the aborted installation for bundles with invalid digital signature [RPF+ 07]. Figure 2.3 presents the life-cycle of OSGi bundles with security management support.

Figure 2.3: Life-cycle of OSGi bundles with security management support

Deployment and Identity-based Cryptography Together with Samuel Galice we worked on another improvement of the process of bundle publication, namely the cryptographic scheme for digital signature. Java Archives and OSGi bundles are usually signed using the SHA-1 and DSA algorithms. We propose to exploit the properties of Identity-Based Cryptography to ease the key management process [PGFU07]. Identity-based cryptography derives the public key of actors - here the bundle signers - from their identity and a seed from a Public Key Generator. Each client can thus verify the validity of any signature without previously having access to its public key certificate: the distribution of PKG parameters to clients is sufficient. This means that only the signers need to contact the PKG. In the case of bundle publication where few signers and numerous clients exists, this makes the key management process lighter. Moreover, through regular regeneration of the signers private key for instance on a daily basis no key revocation is necessary. The main drawback of the Identity-based cryptographic scheme is that the PKG is a single point of failure in the system*: its compromission provides the attacker with sufficient information to forge false signatures. Even though some variants such as the CZK-IBS make possible to prove the forgery a posteriori this weakness may limit the exploitation of IdentityBased Cryptography to protected closed networks rather than exposing such an infrastructure

17

2 Preliminary Development: secure Deployment for the OSGi Platform on the Internet.

2.3 OSGi Bundles as Java Protection Domains Java Protection Domains * Access Control in Java platforms is enforced through Java Permissions. In the context of the OSGi platform code is provided by issuers that do not necessarily know each other and therefore may not trust each other. Neither does the OSGi platform itself trust bundles that are discovered from the environment. It is therefore essential to perform security checks so as to prevent the bundles from executing dangerous methods. Java Permissions are associated with the bundles through Permission Domains. Each bundle is started in the Permission Domain of its signer. For each signer a set of Java permissions is defined in the java.policy configuration file. This mechanism enables the platform administrator to define the minimal set of permissions that are required to execute each bundle. If a bundle intends to perform a sensitive call that is not expressively allowed the call aborts. Java Permissions are defined in the Java 1.2 specification so as to relieve the sandboxing model for Applets: code can be trusted to execute some actions, but not some others [GMPS97]. For instance a local file management application needs access to the file system but not to the network. Since protection domains where not implemented in the OSGi platform we use for implementation, Apache Felix, it was necessary for us to code this feature. Conclusion Figure 2.4 presents the preliminary development that were necessary before the beginning of the actual research work on security models for Java SOP platforms.

Figure 2.4: Overview of preliminary developments The Apache Felix implementation of the OSGi platform is extended with specificationcompliant features: validation of the digital signature of bundles which is part of the OSGi Security Layer and bundle-specific Permission Domains. We also developed a tool for signing and publishing bundles, the SF-Jarsigner which exploit the Open Bundle Repository (OBR) format from the OSGi Alliance. Experiments with Identity-based Cryptography show that this technology can be used with benefits to ease key management in large scale deployment of OSGi platforms. Related software remains proof-of-concept and is not mature enough to be released.

18

Part I

Background and Related Works

19

3

Software Security 3.1 What is Software Se urity ? . . . . . . . . . 3.1.1 Denition . . . . . . . . . . . . . . . 3.1.2 The Domains of Computer Se urity 3.1.3 Software Se urity Assuran e . . . . . 3.2 Se urity Requirement Eli itation . . . . . . 3.2.1 Vulnerability Identi ation . . . . . 3.2.2 Software Se urity Ben hmarking . . 3.3 Te hniques for Software Se urity . . . . . . 3.3.1 Se ure Ar hite ture and Design . . . 3.3.2 Se ure Coding . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

21 21 23 24 26 26 29 32 33 34

The development of secure software is historically based on external protection mechanisms: code sandboxing, black-box testing, as well as on reactive patching. Whereas this approach improves the security* status of applications, it does not tackle the root of unsecurity: applications are mainly not built to be secure, because developers and security practitioners live in different worlds. The dramatic increase of networked applications imply that their are built to withstand attacks* instead of relying on external mechanisms meant to hide their weaknesses. This requires that developers are trained, and that convenient tools are provided to help them create secure applications. This trend lead to the inception of the Software security domain. Its objective is to avoid the ‘patch hell’ and to limit costly upgrades of deployed software.

3.1 What is Software Security ? 3.1.1 Definition Software security is concerned with Black Hat* activities such as developing exploits* and malwares and with White Hat activities which consists in building systems* that resist to attacks from the Black Hat world. Both should be considered as complementary parts [HM04]. Since the goal of this thesis is to define a secure execution platform*, we concentrate here on the White Hat approach. Software Security* is identified as a specific research field by Gary McGraw [McG04] and defined in [McG06]. Its core claim is that security should be built-in into systems1 . 1

https://buildsecurityin.us-cert.gov/

21

3 Software Security Software Security is the technical and research field that is concerned with the realization of provably secure software systems. The objectives of Software Security are listed below: • To enforce security as an intrinsic property of the software, rather than a feature to be added afterwards. • To ensure security properties of all projects, not only highly critical systems such as nuclear plants or aviation and railway control systems; commercial software such as mobile devices or entertainment systems also need to be protected. • To build secure and usable systems [FSH03]. • To make the security level* measurable. • To exploit the full development life-cycle to enhance the security level of the software. Trinity of trouble Security issues in software are due to three factors: • Complexity. • Extensibility. • Connectivity. These features are intrinsic properties of almost all software that is produced in our connected world. They are the core properties of software that provides both a rich user experience and a satisfactory manageability. Pillars of Software Security concepts, or ‘pillars’:

The Software Security approach is characterized by four basis

• Applied risk management [McG06]: software security expenses should be made if and only if they prevent losses that are predictably more important. • Knowledge [McG06] (see Section 3.2): software security should be deduced from a deep understanding of the system and of its weaknesses. • Pro-activity [HL02, VM01]: software security should be performed early in the life-cycle to avoid costly refactoring once the system is deployed. • Software security techniques (see Section 3.3): secure design and coding should rely on well-defined and possibly automated techniques. Topics of Software Security [McG06] ones:

The activities around Software Security are the following

• Code-breaking techniques: identification of vulnerabilities* and related exploits. • Building secure software, e.g. through risk management and security-aware life-cycle. • Designing and using secure languages and platforms, e.g. type-safe languages and virtual machines. • Designing secure software, e.g. through Security Patterns*. • Making sure that software is secure, e.g. through tests, white-box audit and benchmarking. • Educating software developers, architects and users about how to build security in, e.g. by providing security related data in a convenient format.

22

3.1 What is Software Security ?

3.1.2 The Domains of Computer Security Building secure computing systems is not a new topic in itself. However, security is often perceived as a set of features: cryptographic libraries, authentication and authorization libraries, or a property of the execution environment*: hardened OSes, virtual machine sandboxing. The security of the applications themselves often relies on a very intuitive and unorganized approach where developers enforce generic good development practice, but little resource is dedicated to security. This matter of fact is due to 1) the usual scarce resources in software projects which do not enable to perform tasks that do not directly aim at developing new functionalities and 2) the gap between the developer and software engineering community on the first hand and the security community on the second hand. Security development and research is historically performed in following areas: cryptography, network security*, operating system security* and application security*. The first and the last areas deal essentially with providing security libraries. The second and the third deal with with defining stable protocols and systems that are free from vulnerabilities. Figure 3.1 shows the various Security Domains in the common applicative stack for execution environments.

Figure 3.1: The security domains of computer security in the applicative stack Well established approaches obviously do not address one fundamental requirement of today’s software: as far as almost every piece of software is connected to the Internet and thus accessible to hackers, it should be built so as to withstand attacks. Software Security and Network Security The objectives of Network Security [CBR03] is to define secure communication protocols and to control security in the network through firewalls and intrusion detection systems. Its main challenge is to provide protection mechanisms that do not contain vulnerability themselves. Because they introduce more software and are exposed to the Internet, systems like firewalls are a target of choice for hackers. Networkbased mechanisms prove to be efficient against network attacks. Nonetheless, they fail to prevent the exploitation of weak code [McG06]. This is particularly true of high-level firewalls such as HTTP or applicative firewalls which often fail to provide adequate solutions because applicative protocols are more complex and evolutive, as well as less defined, than network protocols. The challenge for Software Security in relationship with Network Security is to provide applications that are not vulnerable to network traffic that can not be protected through firewalls or IDS.

23

3 Software Security Software Security and Operating System Security Operating System Security is the domain of security that deals with the prevention and detection of attacks against an OS. The ultimate goal of such attacks is to ’own’ the OS, i.e. to have full administration rights on it. The challenges of secure OSes are: to build systems that are sufficiently secure for common users, to build hardened OSes such as Security Enhanced (SE) Linux versions [Sri06] or Asbestos [Ste06] and to ensure the integrity of the OS itself through secure bootstrap mechanisms [AFS97]. Alternative to secure OSes are virtual machines or secure middlewares that enable more flexibility in the system design and thus support advanced security schemes (see for instance [VNC+ 06]). Because of their inherent complexity, securing OSes is performed in an empirical manner which questions the fact that OSes can actually be secure at all [THB06]. Attempts at modeling full systems to evaluate their security exist [SSLA06] but remain marginal. Their efficiency can be discussed: vulnerabilities are often very low level features that are abstracted away in most models. The challenge for Software Security in relationship with Operating System Security is to take advantage of system level mechanisms such as isolation in an efficient and configurable manner to exploit them accordingly to the application requirements. Software Security and Application Security Application Security is the domain of security that deals with the prevention and detection of attacks against applications through postdevelopment analysis and protection mechanisms. Its main activities are [McG06] as follows; • • • • • • •

Sandboxing. Black-box testing. Protection against malicious code. Code obfuscation. Executable lock down. Runtime monitoring. Policies management and enforcement.

It considers software as an entity* that should be protected through external measures and focuses on the interactions between the application and the outside world: users, servers, clients*. The Application Security approach faces several challenges. First, it does not provide suitable tools to identify flaws* early in the development life-cycle. This can have important financial consequences because of the cost involved by late bug correction (See Figure 1.1 page 6). Secondly, it does not support the management of security issues in On-the-Shelf (OTS) applications: tools and methods are not available to evaluate efficiently the security status of software outside its development process. Application Security only provides an external view of the software system. Complementing it with Software Security can overcome its limitations by defining the intrinsic properties of the software that ensure the security of the resulting system.

3.1.3 Software Security Assurance The objective of Software Security is to enhance and guarantee the actual security level of programs. It can be achieved through dedicated Software Security Assurance* methodologies which typically enrich software engineering processes with security related activities [GWM+ 07]. The requirement for software security assurance is to define tools, techniques and metrics for building and managing securing software [Bla05, BKF06].

24

3.1 What is Software Security ? Several industrial initiatives aim at integrating software security assurance in the standard development process. Examples are Windows security pushes, back to 2002 [HL02] and the Software Assurance Forum for Excellence in Code (SafeCode) initiative [Sof08] which gathers Juniper, Microsoft, Nokia, SAP, Symantec. Definition Software Security Assurance is ‘the basis for gaining justifiable confidence that software will consistently exhibit all properties required to ensure that the software, in operation, will continue to operate dependably despite the presence of sponsored (intentional) faults’. In practical terms, this means that ‘such software must be able to resist most attacks, tolerate as many as possible of those attacks it can not resist and contain the damage and recover to a normal level of operation as soon as possible after any attacks it is unable to resist or tolerate’ [GWM+ 07]. The NASA [nas92] defines Software Security Assurance as all activities that ensures conformity to requirements, standards and procedures during the software development life-cycle: requirement and specifications, testing, validation and reporting. Secure Development Life-Cycle shown in Figure 3.2.

The Life-Cycle phases of Software Security Assurance are

Figure 3.2: The life-cycle phases of software security assurance They are the following ones [GWM+ 07]: • Requirements for secure software directly affect the likelihood that the software will be insecure. They are directed toward reducing or eliminating vulnerabilities. They are tied to the software development plan and to project management direction. They are not to be confused with requirements for security functionalities that concern access control, identification, authentication and authorization, and cryptographic functions. • Architecture & Design consist in performing a security analysis of the system architecture and of individual components*, as well as in integrating secure design principles and patterns. It should be performed for the whole system as well as for individual modules. • Secure Coding implies a set of issues that should be considered such as language choice, compiler, library and execution environment choice, coding conventions and rules, comments, documentation of security sensitive code and constructs, integration of OTS software and filters and wrappers.

25

3 Software Security • Analysis & Testing represent the most widespread best practices for security assurance. They can be parted in ‘white-box’ techniques such as static analysis, property based testing, fault injection, fault propagation and dynamic analysis of source code, and ‘black-box’ techniques such as binary analysis, penetration testing, fuzz testing and automated vulnerability scanning. It should be performed for the whole system as well as for individual modules. • Distribution & Configuration aim at minimizing the opportunities for malicious actors to gain access and to tamper the software during its transmittal from its issuer to its consumer. Software Security Assurance thus includes activities that are specific to Software Security, but takes issues from the Application Security domain into account to build a coherent system. Representative Secure Development Life-Cycles (SDLC) are the McGraw process for ‘Building Security in’, Microsoft SDLC and the GIAC framework. McGraw process for ‘Building Security in’ [McG06] is built around 7 touch-points: Code Review, Architectural Risk Analysis, Penetration Testing, Risk-based Security Testing, Abuse Cases, Security Requirements Elicitation, Security Operations. Microsoft SDLC [HL02, LH05] consists in enriching the standard development life-cycle with incremental security improvements and emphasizes strongly on developer training. The Global Information Assurance Certification (GIAC)2 framework [McC02] which is based on following principles: integrate security as part of the design, assume a hostile environment, use open standards, minimize and protect system elements to be trusted, protect data at the source, limit access to need-to-known, authenticate, do not subvert in place security solutions, fail securely, log, monitor and audit, and maintain accurate system time and date. Other software security assurance proposals exist, for instance to support component based development [Kim04] or to exploit certification schemes such as the Common Criteria [Llo05]. The AEGIS model [FSH03] is the mapping of security concerns to Boehm’s spiral development model [Boe86]. A comparison of several approaches for software security assurance is proposed by the Software Engineering Institute of Carnegie Mellon University [Noo06]. It emphasizes on the Capability Maturity Model (CMM) which aims at certifying development processes rather than the software itself.

3.2 Security Requirement Elicitation The first step in a software security assurance process is the elicitation of the security requirements. It implies to have a precise knowledge of the vulnerabilities of the system under study and to be able to perform precise benchmark.

3.2.1 Vulnerability Identification The tools for vulnerability identification are taxonomies, Reference Vulnerability Information* (RVI) databases and Top N reminder lists. Attack patterns structure these informations. 2

http://www.giac.org/

26

3.2 Security Requirement Elicitation Vulnerability Taxonomies Taxonomies provide a fine grain description of the properties of each vulnerability. Each taxonomy* should verify the properties of a valid taxonomy as defined by [Krs98] and [HL98]. These properties are the following: objectivity, determinism, repeatability, specificity (disjunction), observability. The seminal works on vulnerability taxonomy have been performed by Abbott [ACD+ 75] and Bisbey [BH78]. The flaws are classified by type of error (such as incomplete parameter validation). This approach turns out not to support deterministic decisions since one flaw can often be classified in several categories according to the context. To solve this problem, Landwehr [LBMC94] defines three fundamental types of taxonomies for vulnerabilities: classification* by genesis of the vulnerability, by time of introduction and by location (or source). Moreover, vulnerabilities should be considered according to specific constraints or assumptions since there existence depends most of the time on the properties of the environment [Krs98]. These assumptions make it necessary to rely on a well defined system model. For instance, such a model is proposed for generic computing systems by the Process/Object Model [BAT06]. Consequently it is difficult for generic purpose databases to rely on specific taxonomies: the Common Vulnerability Enumeration [BCHM99] project has given up the use of taxonomies. Extensive discussions of vulnerability taxonomies can be found in [Krs98] and [SH05].The CWE (Common Weaknesses Enumeration) Project maintains a web page with additional references and a graphical representation of each taxonomy3 . Reference Vulnerability Information (RVI) Databases Extensive databases are meant to maintain up to date references on known software vulnerabilities, to force the system vendor to patch the error before hackers can exploit it. They are also known as Reference Vulnerability Information (RVI), or Refined Vulnerability Information (RVI) sources. Two main types of RVI exists: the vulnerability mailing lists and the vulnerability databases. The main mailing lists are the following: • Bugtraq, 1993 onwards (see http://msgs.securepoint.com/bugtraq/), • Vulnwatch, 2002 onwards (see http://www.vulnwatch.org/), • Full Disclosure, 2002 onwards (see among others http://seclists.org/). The reference vulnerability databases are the following. They are meant to publish and maintain reference lists of identified vulnerabilities. • CERT (Computer Emergency Response Team) Database. It is based on the Common Language for Security Incidents [HL98]4 . • CVE (Common Vulnerabilities and Exposures) Database5 . • CWE (Common Weaknesses Enumeration) Database. It is bounded with the CWE and aims at tracking weaknesses and flaws that have not yet turned out to be exploitable for attackers6 . http://cwe.mitre.org/about/sources.html http://www.cert.org/, Carnegie Mellon University 5 http://cve.mitre.org/, US Department of Homeland Security and Mitre Corporation 6 http://cwe.mitre.org/index.html, US Department of Homeland Security and Mitre Corporation 3 4

27

3 Software Security • CIAC (Computer Incident Advisory Capability) Database7 . • OSVDB, Open Source Vulnerability Database8 . It is centered at Open Source Products. Complementary RVI Sources are the following organizations: SecuriTeam 9 , Packet Storm Security 10 , the French Security Incident Response Team 11 , ISS X-Force 12 , Secunia and SecurityFocus. The limitations of the RVIs is that they follow no stable policy, which makes comparison between sources and between the item of a given sources difficult [Chr06]. Reminder Lists Since catalogs are not so easy to remember and therefore to put into practice, several ‘Top N’ lists have been defined. The motivation for such lists is the recurrent drawbacks of other approaches: vulnerability catalogs do not provide a useful overview of the identified vulnerabilities [Chr06]. One classification of computer security intrusions is given by Lindqvist [LJ97]. It contains external and hardware misuse, and several software misuse cases: bypassing intended control, active and passive misuse of resources, preparation for other misuse cases... The Plover classification13 is an example of rationalization of Vulnerability catalogs to support analysis. It is based on the MITRE CVE Database and contains 300 specific entries that reflect 1400 vulnerabilities identified in the CVE database. Its goal is to suppress redundancy from the original database so as to enable scientific analysis, e.g. using statistical approaches [Chr05]. The Nineteen Deadly Sins of software systems are defined by Michael Howard, from Microsoft [HLV05]. They describe the most common vulnerabilities that are found in enterprise information systems. They concern Web based systems, the architecture of the information systems and the technologies involved. The Open Web Application Security Project (OWASP) maintains a TOP 10 of Web Applications vulnerabilities14 . It concerns input validation, data storage, as well as configuration and error management. Another consortium for Web Application security enforcement, the WASC (Web Application Security Consortium), provides its own threat classification15 . A convenient vulnerability list is provided by Gary McGraw, through the Seven Kingdoms of software vulnerabilities [McG06] [TCM05]. The number 7 is chosen to be easily remembered. Each entry is completed with phyla i.e. precise example of the broader categories that are defined by the kingdoms. The kingdoms are the following: input validation and representation, API abuse, security features, time and state, error handling, code quality, encapsulation + environment. This classification is targeted at enterprise information systems. Attack and Vulnerability Patterns The descriptive spirit of Design Pattern [Ale77], [GHJV94], [MM97], is well suited for application in the security fields, where the question of organizahttp://www.ciac.org/ciac/index.html, US Department of Energy http://osvdb.org/, Open Source 9 http://www.securiteam.com/, Beyond Security Company 10 http://packetstormsecurity.nl/, Non-Profit Organization 11 http://www.frsirt.com/, A.D.Consulting Company 12 http://xforce.iss.net/xforce/alerts, IBM Company 13 http://cve.mitre.org/docs/plover/ 14 http://www.owasp.org/index.php/OWASP_Top_Ten_Project 15 http://www.webappsec.org/projects/threat/ 7 8

28

3.2 Security Requirement Elicitation tion and exploitation of the knowledge is central to the protection of systems. Two types of patterns are defined in the security domain: Attack Patterns and Vulnerability Patterns*. Attack Patterns represent potential attacks against a system. They model the preconditions, process and postconditions of the attack. They can be combined with attack trees, so as to automate the identification of attacks that are actually build from simpler atomic attacks [MEL01]. An extensive presentation of the applications of attack patterns is given in the book by Markus Schumacher [Sch03]. The use of Attack Patterns together with software architecture description to identify vulnerabilities is described by Gegick [GW05]. The limitation of this approach is that the attacks as well as the system must be modelized. This makes the approach impractical and often not realistic based on the available knowledge. The Vulnerability Patterns are used in catalogs of vulnerabilities. They often contain a limited number of informations that are meant to identify the vulnerability, but also to not make it easily reproduceable without a reasonable amount of effort to prevent lazy hackers from exploiting the vulnerability databases as a source of ready-to-exploit attack references. We list here the most wide-spread Vulnerability Patterns, along with the attribute they contain. • Rocky Heckman pattern16 : Name, type, subtype, AKA, description, more information. • CERT (Computer Emergency Response Team) pattern: name, date, source, systems affected, overview, description, qualitative impact, solution, references. • CVE17 (Common Vulnerability and Exposures) pattern: name, description, status, reference(s). • CIAC18 (US Department of Energy) pattern: identifier, name, problem description, platform, damage, solution, vulnerability assessment, references. These Vulnerability Patterns are quite simple ones. They have an informative goal but do not intend to support the reproduction of the vulnerability with a minimum of effort, as other patterns do. This approach makes sense relative to their use context - making users and administrators aware of the existence of the flaws - but are not sufficient to support detailed analysis of the related vulnerabilities.

3.2.2 Software Security Benchmarking The definition of quantified security requirements is based on security benchmarking*. It is based on the assumption that security is not to be considered as an absolute value but should be measurable in order to compare various flavours of a system or a given implementation against a reference implementation. Further references related to security benchmarking can be found on the Security Metrics web page19 . Requirements for security benchmarking research are explicited by the NIST Samate (Software Assurance Metrics and Tool Evaluation) project20 [Bla05]. It is dedicated to improving software assurance by developing methods to enable software tool evaluations, measuring http://www.rockyh.net/ http://cve.mitre.org/ 18 http://www.ciac.org/ciac/index.html 19 http://www.securitymetrics.org/ 20 http://samate.nist.gov 16 17

29

3 Software Security the effectiveness of tools and techniques and identifying gaps in tools and methods. It is complemented by the SRD (Standard Reference Dataset) project which aims at providing a knowledge base for assessing software security tools. SRD requirements to support software security tools evaluation are: • • • • • •

Identify classes of security flaws and vulnerabilities. Develop metrics to assess software. Identify classes of software security assessment* techniques. Document the state of the art in software security assessment tools. Develop measures to evaluate tools. Develop a collection of reference awed or vulnerable programs.

Samate goals are high level ones which express the need to propagate the technical information in industrial software projects. • People: provide education and training. • Process: define life cycle development process, best practices, standards. • Technology: develop new tools. Limitations Metrics and statistics build the basis of security assessment. However, they can not be considered as fully trustworthy data for following reasons [Chr06]: • variation in editorial policy: quantity and type of cataloged vulnerabilities may vary over time, especially for databases where the administration team is evolving; this can lead to inconsistency in the available data. • fractured vulnerability information: each RVI source collects its own data; catalogues may not be coherent. They often can not be translated from one to another. • lack of complete cross referencing between RVI sources: competition between RVIs or simply resource limitation make it not possible to track the data from all (even public) RVIs, so no database can be considered to be comprehensive. • unmeasurable research community bias: researchers vary in skill set, preferences for certain vulnerability types or target product; identified vulnerabilities therefore reflect more the effort spent on analyzing systems than their actual security level. • unmeasurable disclosure bias: vendors and researchers vary in their disclosure models, so again the identified vulnerabilities reflect more the analysis and publication habit that the actual state of the systems. These limitations directly impact the validity of vulnerability catalogs and of the security benchmarking that is based on them. Counter Metrics The first step in security benchmarking is to count occurrences of security related properties. These metrics can be related to the development process or to the code itself [Che06]. Metrics of the first category include the number if identified threats, performed tests, measures of the security-related activities or measures related to the education of the development teams. They can also concern analysis coverage: How much of the code is thoroughly tested ? How much has been reviewed through automated tools ? Through manual review ? Code metrics can measure the number of found bugs, the number of different bugs,

30

3.2 Security Requirement Elicitation the number of occurrences by vulnerability category, by severity, or the number of times a system is mentioned in an RVI. These simple metrics are often used as evolution indicators rather that as absolute values. Moreover, they tend to express the thoroughness of the benchmarking process rather than the actual security status of the target software. For instance, a software that is known to contains an important number of bugs during early development phases is likely to be well tested. Estimation of System Exposure The security level of a software system can be estimated through its exposure. Related metrics are the attack surface*, the approximation of the number of bugs per Line of Code (LoC) and the coverage brought in by protection mechanisms. The attack surface surf is defined as a part of Microsoft SDLC [HPW05]. It is impacted by five dimensions of software systems: • • • • •

targets: processes and data which corruption is the goal of attacks, enablers: processes and data which corruption enables further exploits, channels: means of communicating informations from a sender to a receiver, protocols: the rules for exchanging information associated with a channel, access_rights, that limit the access to the different resource types such as processes, data and channels.

The attack surface is expressed as: surf = f (targets, enablers, channels, protocols, access_rights)

(3.1)

where f can be an additive function that takes the interactions between dimensions into account, or can be weighted to reflect their relative importance. The approximation of Nbug the number of bugs in an application is deduced from experience by considering the number of bugs per Line of Code (LoC) in similar projects [HM04]. Typical values range from 5 bugs/KLoC (1000 LoC) for code that has undergone intensive testing to 50 bugs/KLoC for well-tested code. The approximation Nbug is given by: Size ∗ 5/1000 < Nbug < Size ∗ 50/1000

(3.2)

where Size is the number of LoC (Lines of Code) in the application. Coverage is a metric from the domain of fault tolerance [Arn73]. It expresses the ability of a fault tolerant system to withstand a certain number of faults. The evaluation of coverage is based on the comparison of the values of Mean Time to Failure (MTTF) for the default and the resilient system. To the best of our knowledge, no such metric exists to assess protection mechanisms in the security domain. Estimation of Damage Potential and actual Risk The risk implied by attacks can be expressed as damage potential, or as actual risk [HM04]. The financial risk can then be deduced. Damage potential and actual risk are expressed according to a certain number of factors. The authors suggest to rate the risks as high, medium or low unless more precise measures are available. The metrics depends on following variables:

31

3 Software Security • Attack_P otency, the potential to create damage. High-potency attacks are likely to cause problems that are noticeable by the users. Medium-potency attacks are likely to cause problems that are noticeable by the administrators. Low-potency attacks do not cause noticeable problems. • T arget_Exposure, the measure of how difficult it is to carry an attack. Low-exposure attacks are those that are blocked by a firewall. Medium-exposure attacks are blocked by suitable application configuration. High-exposure attacks can not be blocked through the system configuration. If target exposure can not be measured, it should be estimated to 100 %. • Impact, the measure of the harm the attack causes to the system. For instance, a database that is vulnerable to request injection and fully exposed but contains no data has 0 % of risk to leak private information. If impact can not be measured, it should be estimated to 100 %. The Damage_P otential metric is expressed as: 1 < Attack_P otency < 10 0 < T arget_Exposure < 1.0 Damage_P otential = Attack_P otency ∗ T arget_Exposure/10

(3.3)

The Damage_P otential has a range value from 0 to 1. It can also be expressed as a percentage. The actual risk Actual_Risk brought in by a vulnerability is expressed as: 0 < Impact < 1 Actual_Risk = Damage_P otential ∗ Impact

(3.4)

The Actual_Risk has a range value from 0 to 1. It can also be expressed as a percentage. The financial risk f in_risk bound with a given asset can then be measured: f in_risk = Actual_Risk ∗ Asset_value

(3.5)

with Asset_value the financial value of the protected asset. Typically, protection mechanisms should be implemented when their cost is inferior to the financial risk. Mean Time to Intrusion; Minimum Time to Intrusion More complex metrics exist to express the security properties of a software system such as mean time to intrusion (MTTI) and minimum time to intrusion (MinTTI) [VGMC96]. They are based on the estimation of the skills of potential attackers. They are no absolute measures but relative ones that help compare different versions of the same system.

3.3 Techniques for Software Security The core technical activities for building secure software are architecture and design and secure coding. For critical systems, they can be complemented by certification of the development process or of specific properties of the software.

32

3.3 Techniques for Software Security

3.3.1 Secure Architecture and Design Design Principles The core principles of secure software design are well understood since the publication of the Saltzer and Schröder Principles [SS73]. • • • • •

Economy of mechanism: keep the design as simple and small as possible. Fail-safe defaults: base access decisions on permission rather than exclusion. Complete mediation: every access to every entity must be checked for authorization. Open design: the design should not be secret. Separation of privileges: where feasible, protection mechanisms that require two keys to unlock should be used instead of simpler mechanisms with one single key. • Least privilege: every program and every user of the system should operate using the least set of privileges necessary to complete the job. • Least common mechanism: minimize the amount of mechanism common to more that one user and depended on by all users. • Psychological acceptability: its is essential that the human interface be designed for ease of use, so that the users routinely and automatically apply the protection mechanisms correctly.

Additional requirements have been elicited more recently, in particular to support the specific properties of component-based architectures [GWM+ 07]: • Security-aware error and exception handling: in particular, attacks should not lead to software crash (denial-of-service) or to the release of system-related or applicative data. • Mutual suspicion: components should not trust each other when they are not explicitly intended to. • Isolation and constraint of untrusted processes: misbehavior of un- or less trusted components should not affect the system. • Isolation of trusted/high consequence processes: integrity and availability of sensitive components should be protected by isolating them. Security Patterns The objectives of security patterns is: capture expert knowledge, be domain independent i.e. usable in diverse contexts such as network, operating system, software and application security, and be reusable. They are a direct application of the design pattern concept to the domain of security. A comprehensive survey of security patterns is provided by [YWM08]. The most important collection of security patterns is provided by Kienzle and Elder [KETEH01]. Their report gathers 29 patterns dedicated to web applications and parted in two categories: structural patterns and procedural patterns. Only patterns that are not specific to web applications are presented here. The proposed versatile structural patterns are: hidden implementation, minefield, partitioned application, secure assertion, server sandbox, trusted proxy, and validated transaction. The proposed procedural patterns for the development process are: build the system from the ground up, choose the right stuff, log for audit. Some patterns describe specific security features. These patterns should be seen more as informative as actual reference, but they provide the advantage of supporting explicit definitions of concepts that are not well documented elsewhere in the literature. Patterns for access control are provided by [RFMP06]: Authorization, RBAC, Multi-level security, file authorization, Virtual Address Space access control, reference monitor, session, Single Access Point,

33

3 Software Security Check Point. Patterns for secure development are given by [Rom01]: Authoritative Source of Data, Layered Security, Risk Assessment and Management, 3rd Party Communication, The Security Provider, White Hats* Hack Thyself, Fail Securely, Low Hanging Fruit. Domain specific patterns are targeted at specific environments. Sun Core Security Patterns21 initiative defined patterns for J2EE and enterprise applications [LC05, Bru06]. They are categorize in Web Tier patterns (Authentication enforcer, Intercepting validator...), Business Tier patterns (Container Managed Security, Dynamic Service* Management), Web Service Tier patterns (Message Interceptor, Secure Message Router), Identity Management and Service Provisioning (Single sign-on delegator, Password synchronizer). Sun maintains a list of technologies that implement the patterns. The other domains where numerous patterns are defined is privacy and anonymity [Sch02b].

3.3.2 Secure Coding The objective of secure coding is to enforce language based security. The first principle is to train developers to avoid vulnerabilities and the second one is to embed protection mechanisms in the language through secure language type systems or at least in automated tools. Secure coding is the agile way of implementing software security: responsibility of the code quality is shared among developers. A discussion of the important business stakes related to this topic is to be found at [the08]. Activities of Secure Coding Security issues for the coding activities of the software implementation phase include [GWM+ 07]: • • • • • • • •

language choice, compiler, library and execution environment choices, definition of coding conventions and rules, insertion of comments, documentation of security-sensitive code, constructs and implementation decisions, integration of non-developmental software, identification of filters and wrappers need, and, of course, writing secure code.

Writing secure code is eased by a set of methodologies that can be applied during development, in parallel or between successive development phases: code review, static analysis and security testing. Code Review is performed manually to identify and solve security issues in the code. It must be performed by someone who is not the author of the code to make the code be analyzed as it performs, not as it is intended to perform. Some authors advise reviewers to have a black hat experience so as to better understand the actually threats the software is exposed to [HM04]. The OWASP suggests to perform reviews as coherent sessions organized around a meeting [OWA08]. The process is intended to be lightweight, i.e. the preparation should not exceed the duration of the meeting itself. Roles during a review process are the moderator, who selects the reviewers and conduct the meetings, the authors of the code, the inspectors who 21

http://www.coresecuritypatterns.com/

34

3.3 Techniques for Software Security perform the review, the reader who presents the code during the review meeting and the scribe who records the raised issues during the code review. Each meeting should consist in following steps: • Initialization: the moderator plans the meeting and prepares the review package with the author. The review package contains the source code, the document, review checklists, coding rules as well as other useful material such as the output of static analysis tools. • Preparation: after receiving the review package, the inspectors study the code to search for defects. Typically, the duration of this phase is similar to the duration of the following meeting. • Meeting: the reader presents each code excerpt to the participants. Reviewers bring up issues they identified during the preparation. Ambiguities can also be identified through the (mis)interpretation of the code by the reader. • Correction: the authors address the issues identified and the moderator validates their completeness. Less formal review processes skip either the preparation or the meeting phase. The process can also be iterated again. Another code review process is proposed by M. Howard from Microsoft [How06]. It focuses on the expertise of the reviewer and involves less team interactions. It can also serve as guidelines for the preparation phase of the previous process. • Prioritize. • Review the code. ◦ Rerun all available tools. ◦ Look for common vulnerability patterns. ◦ Dig deep into risky code. These generic processes must be implemented for each target language and system. A strong experience and documentation relative to their security issues is required. Resources can be found for instance for the C language in [HM04] and in the ‘CERT C Secure Coding Standard’22 , for the C++ language with the ‘CERT C++ Secure Coding Standard’23 , for Linux and Unix systems in [Whe03], for Windows-based systems in [HL02], for Java in [Sun07] and for Web application in [OWA08]. Static Analysis supports the automation of code review as well as the verification of complex code properties. It aims at handling the problem of code volume by improving the identification of known vulnerabilities by several orders of magnitude and at letting human reviewers focus on hard issues. Two approaches for static analysis can be identified: production tools and research efforts. The key characteristics of tools for code analysis are the following [McG06]. • Be designed for security: although security analysis is a subset of software quality, the knowledge that underlies it is very specific. 22 23

https://www.securecoding.cert.org/confluence/display/seccode/CERT+C+Secure+Coding+Standard https://www.securecoding.cert.org/confluence/pages/viewpage.action?pageId=637

35

3 Software Security • Support multiple tiers: as applications are often written in several languages and for several platforms, each of these technologies must be supported. • Be extensible: security problems evolve, grow and mutate and tools should adapt accordingly. • Be useful for security analysts and developers alike: analysis output should provide sufficient data for developers to enable them to fix the identified issues; this property makes tools an excellent way of training security-unaware developers. • Support existing development processes and Integrated Development Environments (IDEs). • Make sense to multiple stakeholders: metrics should be provided that support not only secure development but also the business decisions related to the software to be built; release managers, development managers and executives should get useful insight from analysis tools. A wide variety of tools exist. The historical tool for finding bugs is Lint for the C language which was presented by the Bell Labs in 1978 [Joh78]. The better known recent tools are the Rough Auditing Tool for Security24 (RATS) for C, C++, Perl, Php, Python, FlawFinder25 for C and C++, ITS426 [VBKM00] for C and C++, Splint27 for C. MOPS [CW02] applies an original approach of model checking to identify source code defects. Example of commercial tools are Fortify 36028 which supports analysis for C/C++, .NET, Java, JSP, ASP.NET, ColdFusion, ASP, PHP, VB6, VBScript, JavaScript, PL/SQL, T-SQL and COBOL and configuration files, or Coverity Prevent 29 . Two approaches exist for static code analysis: source code analysis and binary analysis. Source code analysis tools support the identification of more complex vulnerability patterns, especially for languages that are compiled into low level binaries such as C. They thus provider a deeper analysis. Binary analysis tools are very useful to assess the quality of code which source code is not available, for instance for COTS. Research efforts reflect the diversity of potential solutions for enhancing static code analysis. Some tools exploit assertions to enrich the set of properties that can be verified such as ESC/Java 30 which exploits the Java Markup Language (JML). Static analysis can be extended to support taint code analysis, i.e. to control the propagation of sensitive data [KZZ06]. It can also be used to validate weak typed languages such as scripting languages [XA06]. Advanced solutions consists in combining static analysis with code injection to optimize runtime security controls [Sch00]: this approach is called ‘security passing style’ (SPS). Experiments have been performed to reduce the overhead of Java security manager. However, proposed implementations do not prove to be faster than the default runtime analysis by Sun. The actual benefit of SPS is the flexibility it introduces in the security models. A powerful approach is to integrate security checks in the language definition itself and to delegates verification to language constructs. An example is provided by the secure type system for Java defined in [Boy04]. http://www.fortify.com/security-resources/rats.jsp http://www.dwheeler.com/flawfinder/ 26 http://www.cigital.com/its4/ 27 http://www.splint.org/ 28 http://www.fortify.com/ 29 http://www.coverity.com/ 30 http://kind.ucd.ie/products/opensource/ESCJava2/ 24 25

36

3.3 Techniques for Software Security The collateral consequence of static analysis is that it enables a seamless training of the developers, since identified vulnerabilities are documented so as to ease the correction process. Testing is the activity that validates the security properties of a given software system. A comprehensive framework is defined by the OWASP Testing Guide [OWA07]. Security testing should be based on following principles: • There is no Silver Bullet: no single tool can make software secure, • Think strategically, not tactically: tests should be performed during the initial development of software as much as possible; the cost overhead is by far reduced when compared to the classical wait-and-patch approach to security, • The SDLC is king: developers are comfortable with classical development life-cycles; security should be integrated to limit organizational overhead, • Test early and test often, • Understand the scope of security: according to the considered project, • Mindset: security testers should think ‘outside the box’, not only in term of foreseen functionalities, • Understanding the subject: the application should be well documented and tests performed according to it, • Use the right tools: in order to increase efficiency, • The devil is in the details: every section of code should be reviewed, • Use source code when available, • Develop metrics: to monitor the evolution of the project. Testing processes can be either integrated in a V life-cycle [OWA07] or performed in an agile manner using abuse cases [McG06] as reference. Conclusion Software Security provides a set of methods and tools that can be integrated in the classical software development life-cycle to enhance and control the security level of built systems. By preventing vulnerabilities rather than patching them, it promises to complement in a powerful way network security, because firewalls have a hard time in preventing the access to flawed applications, operating system security, since security responsibility is transfered to a certain extent from the administrator to application developers and application security by making software not only robust through a protective shell but secure by design and by implementation.

37

3 Software Security

38

4

Security for Java Platforms 4.1 The default Se urity Model . . . . . . . . . . . . . 4.1.1 The Java Language . . . . . . . . . . . . . . 4.1.2 Byte ode Validation . . . . . . . . . . . . . 4.1.3 Modularity . . . . . . . . . . . . . . . . . . 4.2 Vulnerabilities . . . . . . . . . . . . . . . . . . . . . 4.2.1 Vulnerabilities in the Java Virtual Ma hine 4.2.2 Vulnerabilities in Code . . . . . . . . . . . . 4.3 Se urity Extensions . . . . . . . . . . . . . . . . . . 4.3.1 Platform Extensions . . . . . . . . . . . . . 4.3.2 Stati Analysis . . . . . . . . . . . . . . . . 4.3.3 Behavior Inje tion . . . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

40 40 41 42 45 45 46 50 50 52 54

The Java environment is composed by two main parts: the Java language specified in [GJSB05] and the Java virtual machine specified in [LY99]. It is designed with the assumption that no software entity* is to be trusted and therefore that each need to be checked. The first success of Java was the inception of Java Applets which enabled fully untrusted code provided by unknown web sites to be executed in a browser [DFW96]. This feature demonstrated the isolation brought by the Java Virtual Machine (JVM) between the applications and the underlying operating system. It is made possible because each internal unit of the JVM is designed to enforce such security*. Figure 4.1 shows the architecture of a Java Virtual Machine (adapted from [COR07]).

Figure 4.1: Architecture of a Java Virtual Machine The units of the JVM are:

39

4 Security for Java Platforms • SSU : the System Services Unit. It contains the class loader and Bytecode verifier, the thread management, management and debugging tools and timers. • MMU : the Memory Management Unit. It contains the reference handling, finalization, garbage collection and fast allocation mechanisms. • ExecU : Execution Unit. It contains the exception handling, Just-In-Time (JIT) Compiler, Interpreter and JNI mechanisms. It is responsible for executing runtime operations. • OS Virtualization* Layer Unit: a platform*-independent abstraction layer to access host resources. It is the very module that is in charge of application virtualization. The default Java security model is presented. Its sound design does not prevent the existence of vulnerabilities* in the JVM and in the code of Java applications. Security extensions to the JVM are then detailed. They intend to address the challenges that are brought in by the simultaneous execution of mutually untrusted components*.

4.1 The default Security Model The security model of Java evolved from HotJava, to Netscape, to Java 2 [DFW96]. It was first designed to support the secure execution of Applets, which are potentially malicious code pieces. This is made possible by the strict language definition and its mechanisms for security enforcement. The shift towards components applications urged the support of modularity*: components from various sources can be executed on the same platform with no mutual access to their data. A comprehensive introduction is to be found in [GED03].

4.1.1 The Java Language The Java language is meant to be a safe subset of the C language. This means that all language elements that can be exploited in C programs such as the nefarious buffer overflows or memory management activities are removed from the specification. This simplicity intends to free developers from error prone tasks and to enhance their productivity by letting him focus on the features they develop. The principles that underlie the security of the Java language are the following [GM96]: • • • •

Type safety. Automated memory management. Bytecode validation. Modularity.

Type Safety means that each language element has a given type and can only perform actions that are coherent with this type. In particular type safe languages are free from pointers which enable to manipulate memory addresses independently of their type. Type safety is enforced at three different moments: during the compilation, when the Bytecode is loaded into memory, and at runtime to ensure type coherence with libraries that are dynamically linked. Some authors consider that the Java language is not completely type safe since it enables to handle Sets and Collections of data without regard to their actual type. Errors then lead to thrown exceptions at runtime. The solution at the language level is the use of Generics which force the data structures to only contain objects of a given type. For backward compatibility

40

4.1 The default Security Model reason, this feature is not made mandatory. A solution exist in certain SOP platforms such as Guice, that create objects in an ‘extraordinarily type safe’ manner (see Section 5.2.2) and thus prevent runtime exceptions which could occur due to cast errors. Automated Memory Management consists in hiding memory allocation and de-allocation from the language. This is done by postponing these operations from the compilation phase as in C and C++ to runtime. The virtual machine is itself responsible to allocate required memory at object instantiation and to release the memory that is not longer used through a Garbage Collector (GC). Garbage collection is performed by maintaining a list of objects in use. When an object is dereferenced by all its callers, it can be removed safely. This prevents development errors. It also prevents malicious code from directly accessing the memory, which makes the resulting code more robust against attacks*. Additional mechanisms are enforced to guarantee the integrity of the memory. For instance, array bounds are checked to prevent reading or writing data outside the defined data structures. This prevents the buffer overflows and therefore eliminates one important attack vector* used for executing malicious code. Bytecode Validation mechanisms are presented in Section 4.1.2. They guarantee that the executed Bytecode is compliant with the specification and therefore prevent the forgery of instruction suites that could not be derived from the source code by a valid compiler. Modularity mechanisms are presented in Section 4.1.3. They enable to execute several components with no naming conflict and unintended access. In particular, they target classes that are loaded from diverse locations such as remote code servers. The security mechanisms for the Java language mix the Software Security*, e.g. language features, and the Application Security* approaches in a much integrated way such as modular support. It is a powerful illustration of the complementarity of the two techniques: Software Security is enforced when possible to assert comprehensive security properties in an efficient manner. When it is not sufficient, it is complemented with Application Security mechanisms.

4.1.2 Bytecode Validation Loaded Bytecode is considered as untrusted. It must therefore be completely validated before its execution to ensure it is compliant with the specification. This validation occurs in the Bytecode verifier of the JVM immediately before the code is loaded in memory [GM96]. It is complemented by runtime checks in two cases: when the required information is not available during validation, e.g. when several libraries are used together and when optimizations make runtime checks more efficient. Figure 4.2 shows the process of Bytecode verification. The code is generated from a .java source file to a Bytecode .class file by the compiler. Language specifications are enforced here but no guarantee exists that 1) the compiler is valid [Tho84], 2) the Bytecode has not been forged without compiler or 3) the file is not modified during its transfer through the network or any other medium. When it is loaded, the Bytecode is therefore first checked in the System Service Unit (SSU) of the JVM, before being forwarded to the Execution Unit (ExecU) for JIT compilation or interpretation. The validation that is performed by the Bytecode verifier is made of following tasks [Sch02a]: • Structural correctness: checks attribute lengths, class file validity.

41

4 Security for Java Platforms

Figure 4.2: The process of Bytecode verification • Data type correctness: checks that final classes are not subclassed and final methods not overridden, that all classes except java.lang.Object have a superclass, that all fields and methods references have legal name, classes and type signature. • Bytecode checks: checks 1) static constraints: control flow correctness, validity of exception handlers; 2) static constraints: reachability of subroutines and exception handlers, data flow validity such as absence of buffer overflow and underflow. • Symbolic reference management during runtime: checks the validity of current class references, type safety of method calls and field access, visibility of method calls and field access. This approach for Bytecode validation pertains to the category of Software Security, because properties that make the code secure are deeply embedded in it. However, it is a very unflexible method that can only enforce permanent properties.

4.1.3 Modularity Support for modular programming intends to enable the execution of mutually unknown components that do not necessarily trust each other. Class Loaders built a subsystem of the JVM that is responsible for finding classes and making them available for execution1 . They are organized as a hierarchy, as shown in Figure 4.32 . The presence of application specific class loaders is optional but required to support namespace isolation between the components. When executed in its own class loader, a component have access to the classes of its parent class loaders as well as to its own classes. This ensures that no naming conflict can occur if several components contains classes with the same 1 2

http://interviewjava.blogspot.com/2007/04/what-is-class-loader-and-what-is-its.html, read on 2008/08/18 http://java.sun.com/docs/books/tutorial/ext/basics/load.html, read on 2008/18/08

42

4.1 The default Security Model

Figure 4.3: The hierarchy of class loaders

name. As shown in Figure 4.3, the component 1 and 3 can both have an internal class named my.package.Foo that have different implementations. They will share system classes such as java.lang.System. Each class loader have the following behavior. It first checks whether the class to be loaded is already available. If this is the case, it returns it. Otherwise, it delegates the search for the new class to its parent class loader. If this request is unsuccessful, i.e. if none of the class loader in the hierarchy finds the class, the current class loader searches for the class itself with the findClass() method. The default class loaders of the hierarchy are the bootstrap class loader, the extension class loader and the system class loader. The bootstrap class loader has access to the classes that are required to launch the JVM. In particular, it is responsible for the runtime classes (rt.jar archive) and the internationalization classes (i18n.jar archive). The extension class loader is responsible for extension classes such as the implementation of cryptographic libraries. It concerns the archive that are stored in the lib/ext/ directory such as the sunjce_provider.jar, sunpkcs11.jar or localedata.jar archives in the case of the Sun JVM. The system class loader is responsible for the classpath, which is set through the CLASSPATH environment variable or the -classpath or -cp option of the JVM. Additional class loaders can be defined by the applications or by the framework used. For instance, the OSGi framework loads each bundle* in a dedicated class loader. Classes can be shared between the bundles and thus between the class loaders through a specific package import and export mechanism. The obvious security benefit of class loaders is the namespace isolation which prevents one component to access another without being explicitly allowed to. Moreover, system classes are shared which prevent inconsistencies. However, class loaders are not designed to support strong isolation. In particular, static

43

4 Security for Java Platforms variables in parent class loaders are shared. For instance, the System.out variable in the bootstrap class loader can be accessed and modified by all classes. This can lead to unrequired interference such as denial-of-service or to data transmission that should not occur. Similarly, static synchronized calls can be exploited. Moreover, they do not support resource isolation which is the second necessary protection besides namespace safety. The last drawback of class loaders is that they can not be simply removed to uninstall a component since this could imply inconsistencies in programs executed by others. For all these reasons, class loaders build an interesting feature of Java component platforms* but can not be considered to be secure. Protection Domains * are the solution proposed by Sun for securely executing components in parallel. This approach is also known as Stack Based Access Control (SBAC) since it relies on the inspection of the call stack to determine whether all callers have sufficient rights to execute a given method. It is defined in [BG97, WF98]. A technical presentation is to be found in [Sun]. Per se, a Protection Domain is the set of objects currently directly accessible by a principal* [SS73]. A Principal is the software representation of an entity (an individual, a corporation, a login, a place in the file system) to which execution rights are granted. In the case of Java component platforms, a Protection Domain thus represents all objects that can be accessed by one component. It is made of all unsensitive code which can be called seamlessly and of sensitive code to which access is granted explicitly through a permission policy. The use of protection domains in Java applications requires that the security manager of the JVM is enabled and the policy file defined according to the requirements of the applications: java -Djava.security.manager -Djava.security.policy=java.policy ... Security checks are performed at runtime in the code of the application, the framework or the standard libraries through a call to the security manager: mySecurityManager.checkPermission( new ServicePermission(serviceName, ServicePermission.GET));} The example shows the permission check in the OSGi platform when a service* is requested. The security manager then performs following operations: it reconstructs the call stack that leads to the sensitive call and identifies related principals; it checks whether all principal have sufficient rights to perform the sensitive calls. If these rights are not granted, a java.lang.SecurityException is thrown. One of the main limitations of protection domains is the important overhead they imply, especially when their number and the frequency of security checks increases. An important improvement effort has been dedicated to the original implementation [GS98]. Alternative solutions, e.g. through static analysis, have been proposed. However, they remain research efforts and have not succeeded at providing more efficient solutions. Evaluation of Java Security After its release, the Java 2 language and platform have been subject to an extended analysis by the industrial and academic community [CM99]. The most important discussion arose about the conservation of the property of type safety in the presence of several class loaders. A subset of the Java language has been proved to be type safe [DE97], before an implementation error in the class name resolution was identified [Sar97].

44

4.2 Vulnerabilities This error has been formally modeled afterwards [JLT98] and corrected in later releases of Sun JVM. Moreover, following limitations of the Java Stack Inspection policy are identified in the literature [ES00]: • If a new thread is created from within a doPrivileged block then that thread will continue to enjoy amplified privileges, even though its code might not be within the scope of a doPrivileged block and even after its creator has exited from within the doPrivileged. This is because the new thread starts execution with a copy of its creator’s call-stack (whose top frame is marked as being within the scope of a doPrivileged). • When a class B extends some class A but does not override A’s implementation of a method foo(), then the protection domain for A (and not B) will always be used by checkPermission for foo’s stack frame. Because B can extend A in ways that may act the semantics of foo, (such as by overriding other methods), one might argue that the wrong protection domain is being consulted.

4.2 Vulnerabilities Vulnerabilities in Java-based systems* can be found in two distinct locations: the virtual machine and application code. Their disclosure acts as a strong incentive for both JVM and application developers to increase the security level* of the system.

4.2.1 Vulnerabilities in the Java Virtual Machine Generic JVM Vulnerabilities can be identified. A comprehensive analysis of bug databases of Sun JVM and Jikes3 is proposed by [COR06]. They can be exploited to perform at least denial-of-service exploits*. Statistics related to the origin of failures are extracted from a selected set of bug entries: 103 bugs from Sun and 28 bugs from Jikes. Consequently, only those two VMs are directly concerned by the given results. The conclusions are the following: • Garbage collection is responsible for 73 % of the failures in the memory management unit (MMU). • Runtime support operations such as method invocation, stack frame allocation and deallocation, exception handling and optimized Just-in-Time (JIT) compilation are responsible for 77 % of the failures in the Execution Unit (ExecU). • Thread management is responsible for 76 % of the failures of the System Service Unit (SSU). • Most of the failures (80 %) occur when the JVM is running under an important workload. JVM optimizations such as garbage collection and JIT compilers are clearly identified as reliability bottlenecks. A clear trade-off appears here between performance and reliability. These JVM failures worsen when the applications are running a long-time without interruption. This phenomenon is called aging and is especially present for application servers. It leads to the following behavior of an example mail application running on a Java server: 3

http://jikesrvm.org/

45

4 Security for Java Platforms • A consistent memory depletion trend (up to 50 KB/min) during periods of low garbage collector activity. • A consistent throughput loss (up to 2.4 KB/min) has also been observed. • The Just-In-Time compiler is responsible for a not negligible memory depletion trend (numerical value is not given). The experiment is conducted with a mail server called James for a duration of 100 hours with a workload generator. More exotic vulnerabilities can be exploited. For instance, memory errors can be used to abuse a virtual machine [GA03]. The attack relies on an aleatory error in a program with a very repetitive address scheme. The shift of any bit in one method leads to type unsafe behavior. The attack is sped up through physical access to the machine: the authors use a simple light bulb to perform the bit shift. Moreover, each specific implementation of the JVM introduces new vulnerabilities. Mobile platforms are especially fragile to such attacks since they are meant to be extended through MIDlets provided by third parties. A vulnerability analysis is proposed for Sun Connected Limited Device Configuration (CLDC) by [DSTZ05]. Following flaws* are identified: SMS can be sent by malicious MIDlets, network and cryptographic errors exist in the MIDP Reference Implementation (MIDP RI), data is recorded in stores not protected from malicious attacks, no quota can be set in data storage, low level helper functions* for high level libraries are available for execution by MIDlets, and MIDlets can be transferred from a device to another by the user. These vulnerabilities can not be extrapolated to other VMs or other implementations. However, they are representative of the type of vulnerabilities that can occur in Java environments.

4.2.2 Vulnerabilities in Code Various authors propose lists of vulnerabilities that are to be avoided and coding rules that help prevent them. Though these lists are partially overlapping, they each provide specific entries. Source Code The first set of vulnerabilities that can be identified in the Java source code consists in counterintuitive behaviors of Java programs [Blo01, BG05] known as puzzlers. They concern the range limit of numbers or intricate exception and thread management. These ‘strange’ behaviors lead to code misunderstanding and to infinite loops or program deadlocks. An example of such an infinite loop is given in Listing 4.1. Its origin is the way the integer counter is incremented: when it reaches Integer.M AX_V ALU E, the count goes on with Integer.M IN _V ALU E. Listing 4.1: Example of a counterintuitive infinite loop in the Java language p u b l i c s t a t i c f i n a l i n t END = I n t e g e r .MAX_VALUE; p u b l i c s t a t i c f i n a l i n t START = END − 1 0 0 ; // or any o t h e r s t a r t v a l u e f o r ( i n t i = START; i