textfiles-politics/politicalTextFiles/ncsarev.txt

NCSA POLICY CONCERNING SECURITY PRODUCT REVIEWS
February 17, 1990.

	Purpose: NCSA product reviews are intended to present
complete, thorough, useful reviews of security products
to the members of the NCSA.  This document's purpose is
to set forth the NCSA policy concerning such reviews.
This policy is open for discussion.

	Reviewers: Reviewers may be single individuals or
"review teams."  Reviewers should have some knowlege of
the application of the product, and should be capable of
writing detailed reviews.  In the case of review teams,
the teams may consist of expert users, as well as novice
users.  The role of the novice user is to provide input
on product ease-of-use and quality of documentation.

	Conflict of Interest: NCSA reviewers must have no
interest in the product reviewed which would compromise
the integrity or accuracy of the review.  All reviews
will be signed by their authors.

	Procurement of Products: Products may be solicited
directly from manufacturers/software houses on behalf of
the NCSA.  In return for a free evaluation copy, the
product review will become a permanent part of the NCSA
BBS, available for viewing by all members.  After
completion of the review, the reviewer shall be granted
the license to the product.

	Evaluation Copies: No review will be performed on a
copy which is limited in function. No review will be
performed on a "beta" version of a product, or any
product which is not available to the product.

	Limit of Liability: The NCSA shall assume no
liability for, or make claims of, the capabilities or
fitness of any products.  All reviews shall be carried
out to the best ability of the reviewer/review team, and
be edited if necessary by the NCSA staff.

	Comments/Clarifications/Rebuttals: After a product
has been reviewed, the review shall be posted on the
NCSA BBS, and the manufacturer be allowed to comment on
the review for a period of 60 days.  A copy of the
review will also be sent to the manufacturer for their
comment. After such time, the review will be edited if
necessary, based upon the responses of both the
manufacturer and any others who have commented.  The
review will then become part of the permanent library of
the NCSA.  A summary may be placed in the NCSA
newsletter; the full review will be placed on the NCSA
BBS for downloading by members.

	Classifications: A detailed system of classification
shall be developed to assist both reviewers and readers
in their respective efforts.  For example, such
categories might include PC Access Control, Data
Encryption, Virus Detection, etc.

	Review Outline: The reviewer(s) shall follow the
review outline presented at the end of this document. 
In this way, similar products can be compared directly.

	Comparative Reviews: Where possible, a single review
will comprise a category of products. As each new
product within that category is reviewed, the new review
will be merged with the existing reviews. Where
possible, tables will be created comparing products. 
This will be done to aid members in choosing a product.

	Product Classification Overviews: In cases where
there are many products in a single category, a review
team may be assigned to evaluate all the reviews and
pick an "NCSA Choice".  This would be the NCSA's
official recommendation, and would be awarded to the
product that best meets the criterion for its category.

	Quantitative Ratings: A system of ratings shall be
developed, in order to more easily compare products.  At
the time of review, an NCSA security rating will be
assigned.  This will consist of a number from 0.0 to
10.0, with 0.0 providing the least security, and 10.0
the most.  A scale shall be developed to aid both
reviewers and readers compare scores (i.e. 6.0-8.0
Average 8.0-10.0 Recommended, etc.). The exact form of
these ratings will be developed over time, as the first
reviews are conducted.

	Access to Reviews: Reviews shall be placed in a
restricted area of the NCSA BBS, to enable only dues-
paying members to have access.  Hard copies of the
reports may be requested for a small fee.

	Review for Fee: At a manufacturer's or member's
request, NCSA will review a specified product. A fee may
be charged for such review, but this fee will in no way
affect the nature of the review.

	Review Content: Each review will contain the
following information:
	* Reviewer(s) name
	* Product name
	* Version of product reviewed (version number and/or
date)
	* Product pricing information
	* Manufacturer name, address, phone.
	* Product category/function.
	* Product description.  This description will have a
heavy emphasis on the security offered by the product,
even if security is not the main focus of the product.
	* Product capabilities. What specific features the
product offers. Such information may be drawn from
marketing materials, but must be verified by the
reviewer.  Such narrative might be presented in bullet
or other narrative format.
	* Definition of categories used in the ratings, and
general rating approach. This definition will be
sufficiently explicit that other reviewers will be able
to apply the method and obtain the same results on this
product. Examples of categories likely to be included:
ease of installation, ease of use, degree of protection
offered, adequacy of documentation, support, accuracy of
manufacturer's claims concerning the product, overall
value.
	* Category ratings, with justification.
	* Summary of ratings, in tabular form.

	About this document: The first draft of this document
was prepared by Charles Rutstein, co-sysop of the NCSA
BBS. David Stang revised it.  Comments are invited. 
Write NCSA, Suite 309, 4401-A Connecticut Ave. NW,
Washington DC 20008. Or call NCSA voice 202-364-8252 or
leave a comment to the SYSOP on the NCSA BBS: 202-364-
1304.