<テクニカルレポート>
A TURING Test Scenario to Estimate an AI System's Validity

作成者
本文言語
出版者
発行日
収録物名
出版タイプ
アクセス権
関連DOI
関連URI
関連情報
概要 There is an urgent need to develop, to implement and to apply methodologies resp. tools for complex system validation. The inability to evaluate complex systems may become a crucial limiting factor to... future technological progress. The authors are engaged in a quite comprehensive endeavour towards intelligent system validation. This paper deals with both the fundamentals of a so-called "TURING Test Methodology" for intelligent system validation and some ideas to make them practicable. First, we survey several concepts of verification and validation. Our favoured concepts are lucidly characterized by the words that verification guarantees to build the system right whereas validation deals with building the right system. Next, we critically inspect the thought-experiment called the TURING test. It turns out that, although this approach may not be sufficient to reveal a system's intelligence, it provides a suitable methodological background to certify a system's validity. The principles of our validation approach are surveyed. The informal discussion leads to a collection of a few formalisms suitable for a systemtic approach to intelligent system validation. A discussion of essential problems beyond the basic TURING test methodology reveals some problems in making it practicable. Even in very simple scenarios the problem of vagueness and uncertainty in the experts' knowledge has to be managed in a reasonable way. This is due to the fact that the real target system's behaviour is not available as a yardstick. Instead, experts' knowledge has to be taken as a basis for system validation. The problem of measuring a system's behaviour against some knowledge source as insecure as human beings is analyzed. There is motivated, developed and illustrated a family of TURING test scenarios which provides the basis of further steps towards systematic intelligent system validation. The TURING test scenario presents some main ideas how to perform a (more or less) good approximation of the target knowledge by using some (more or less competent) experts' knowledge. Based on the precise formal concepts developed before, individual steps of performing the TURING test scenario can be described in some detail. The test results need to be evaluated to certify a given intelligent system's validity. Formal concepts can be utilized to estimate competence, local validity, and global validity.続きを見る

本文ファイル

pdf trcs133 pdf 420 KB 192  
gz trcs133.ps gz 241 KB 46  

詳細

レコードID
査読有無
タイプ
登録日 2009.04.22
更新日 2018.08.31