ELECTRONIC RECORDS MANAGEMENT

• Citation: 36 CFR Part 1236
• Sections: 26

§ 1236.2 Definitions that apply to this part.

(a) See § 1220.18 of this subchapter for definitions of terms used throughout Subchapter B, including part 1236.

(b) As used in part 1236—

Administrative metadata are elements of information used to manage records and relate them to one another. Administrative metadata elements describe how a record was created, any access and use restrictions that apply to it, information about the record series to which it belongs, and the disposition schedule that identifies its retention period.

Checksum is a value that is computed on data and is used to authenticate information by indicating when a file has been corrupted or modified. This value is also called a “hash value,” “hash code,” “digest,” or simply “hash.”

Descriptive metadata are elements of information that describe the records or set of records itself. They apply to both the source records and any versions produced through digitization. Descriptive metadata for individual source records include such elements as the title of a record, a description of its contents, its creator, and the date it was created. These elements support searching for and discovering records.

Digitizing is the process of converting paper or analog records into electronic records.

Electronic information system means an information system that contains and provides access to computerized Federal records and other information.

Electronic mail system means a computer application used to create, receive, and transmit messages and other documents. Excluded from this definition are file transfer utilities (software that transmits files between users but does not retain any transmission data), data systems used to collect and process data that have been organized into data files or data bases on either personal computers or mainframe computers, and word processing documents not transmitted on an e-mail system.

Embedded metadata are textual components that exist alongside the content (usually binary data) within the file. Embedded metadata may be used to make self-describing digital files that contain administrative rights, and technical metadata and can be appropriately managed outside of a recordkeeping system.

Intellectual control is the information necessary to identify and understand the content and context of the records.

Media are the physical forms on which records are stored, such as paper, photographs, compact discs, DVDs, analog tapes, flash drives, local hard drives, or servers.

Metadata consists of preserved contextual information describing the history, tracking, and/or management of an electronic document.

Mixed-media files are records in different forms of media. A file, when used in the phrase “mixed-media file,” is a group of records—regardless of location and type of media—that belong together or relate to a topic. For example, a mixed-media case file could be a box with paper notes, audio recordings of interviews, and a CD of photographs, along with physical evidence stored separately in an evidence locker. Records in a file may be in more than one media type because of how agencies create, maintain, and use records, shifts in technology, and the topic or activity involved.

Physical characteristics of source records include the media, the method that information is recorded, the physical condition of the material, and the smallest level of detail present. The physical characteristics of records printed on paper include: the type of paper (office paper, Thermofax, photographic print); the type of printing (laser printed, fax printed, typewritten, half-toned, handwritten); appearance (color, inks, continuous tone or monochrome images); size, and other methods of conveying information (embossed seals, stamps). These traits determine the methods and equipment used to digitize records.

Physical control is having the information necessary to physically manage records. This includes knowing where the records are housed, whether any records are missing or stored separately, and the records' physical form (media types, the records' dimensions, and the physical characteristics).

Project plan is a document that identifies the records that are to be digitized, an estimate of their volume and of the media types that are present, the image quality parameters selected to capture necessary information, the date range of the records, a copy of the applicable agency records schedule(s); any indexes used to maintain intellectual and physical control; and a quality management (QM) section that describes quality assurance (QA) objectives, quality control (QC) procedures to identify and correct errors during digitization, and the QC reports that will be used to identify and remediate errors when detected.

Quality assurance (QA) refers to proactive QM activities focused on preventing defects by ensuring that a particular product or service achieves certain requirements or specifications. A QA program is heavily dependent on QC data to search for patterns and trends. QA activities also include controlled experiments, design reviews of digitization workflows, and system tests. QA programs can improve quality by creating plans and policies or by creating and conducting training.

Quality control (QC) refers to QM activities that examine products through inspection or testing to determine if they meet predetermined specifications. The purpose is to detect defects (deviations from predetermined requirements) in products or processes.

Quality management (QM) refers to the overall management functions and underlying activities that determine quality policies, objectives, and responsibilities, and that implement them through planning, control, assurance, and improvement methods within the quality system.

Technical metadata are elements of information that describe the properties of computer files, the hardware used to create them, and the parameters used by systems to render them. Technical metadata may include elements such as a file's byte size, file format and version, color encoding, and the type of equipment used to make the file (for example, camera name or scanner manufacturer).

Unstructured electronic records means records created using office automation applications such as electronic mail and other messaging applications, word processing, or presentation software.

Validating is the process of ensuring that the records meet the requirements of this part.

§ 1236.4 What standards are used as guidance for this part?

These regulations conform with ISO 15489-1:2001. Paragraph 9.6 (Storage and handling) is relevant to this part.

§ 1236.6 What are agency responsibilities for electronic records management?

Agencies must:

(a) Incorporate management of electronic records into the records management activities required by parts 1220-1235 of this subchapter;

(b) Integrate records management and preservation considerations into the design, development, enhancement, and implementation of electronic information systems in accordance with subpart B of this part; and

(c) Appropriately manage electronic records in accordance with subpart C of this part.

§ 1236.10 What records management controls must agencies establish for records in electronic information systems?

The following types of records management controls are needed to ensure that Federal records in electronic information systems can provide adequate and proper documentation of agency business for as long as the information is needed. Agencies must incorporate controls into the electronic information system or integrate them into a recordkeeping system that is external to the information system itself (see § 1236.20 of this part).

(a) Reliability: Controls to ensure a full and accurate representation of the transactions, activities or facts to which they attest and can be depended upon in the course of subsequent transactions or activities.

(b) Authenticity: Controls to protect against unauthorized addition, deletion, alteration, use, and concealment.

(c) Integrity: Controls, such as audit trails, to ensure records are complete and unaltered.

(d) Usability: Mechanisms to ensure records can be located, retrieved, presented, and interpreted.

(e) Content: Mechanisms to preserve the information contained within the record itself that was produced by the creator of the record;

(f) Context: Mechanisms to implement cross-references to related records that show the organizational, functional, and operational circumstances about the record, which will vary depending upon the business, legal, and regulatory requirements of the business activity; and

(g) Structure: controls to ensure the maintenance of the physical and logical format of the records and the relationships between the data elements.

§ 1236.12 What other records management and preservation considerations must be incorporated into the design, development, and implementation of electronic information systems?

As part of the capital planning and systems development life cycle processes, agencies must ensure:

(a) That records management controls (see § 1236.10) are planned and implemented in the system;

(b) That all records in the system will be retrievable and usable for as long as needed to conduct agency business (i.e., for their NARA-approved retention period). Where the records will need to be retained beyond the planned life of the system, agencies must plan and budget for the migration of records and their associated metadata to new storage media or formats in order to avoid loss due to media decay or technology obsolescence. (See § 1236.14.)

(c) The transfer of permanent records to NARA in accordance with part 1235 of this subchapter.

(d) Provision of a standard interchange format (e.g., ASCII or XML) when needed to permit the exchange of electronic documents between offices using different software or operating systems.

§ 1236.14 What must agencies do to protect records against technological obsolescence?

Agencies must design and implement migration strategies to counteract hardware and software dependencies of electronic records whenever the records must be maintained and used beyond the life of the information system in which the records are originally created or captured. To successfully protect records against technological obsolescence, agencies must:

(a) Determine if the NARA-approved retention period for the records will be longer than the life of the system where they are currently stored. If so, plan for the migration of the records to a new system before the current system is retired.

(b) Carry out upgrades of hardware and software in such a way as to retain the functionality and integrity of the electronic records created in them. Retention of record functionality and integrity requires:

(1) Retaining the records in a usable format until their authorized disposition date. Where migration includes conversion of records, ensure that the authorized disposition of the records can be implemented after conversion;

(2) Any necessary conversion of storage media to provide compatibility with current hardware and software; and

(3) Maintaining a link between records and their metadata through conversion or migration, including capture of all relevant associated metadata at the point of migration (for both the records and the migration process).

(c) Ensure that migration strategies address non-active electronic records that are stored off-line.

§ 1236.20 What are appropriate recordkeeping systems for electronic records?

(a) General. Agencies must use electronic or paper recordkeeping systems or a combination of those systems, depending on their business needs, for managing their records. Transitory e-mail may be managed as specified in § 1236.22(c).

(b) Electronic recordkeeping. Recordkeeping functionality may be built into the electronic information system or records can be transferred to an electronic recordkeeping repository, such as a DoD-5015.2 STD-certified product. The following functionalities are necessary for electronic recordkeeping:

(1) Declare records. Assign unique identifiers to records.

(2) Capture records. Import records from other sources, manually enter records into the system, or link records to other systems.

(3) Organize records. Associate with an approved records schedule and disposition instruction.

(4) Maintain records security. Prevent the unauthorized access, modification, or deletion of declared records, and ensure that appropriate audit trails are in place to track use of the records.

(5) Manage access and retrieval. Establish the appropriate rights for users to access the records and facilitate the search and retrieval of records.

(6) Preserve records. Ensure that all records in the system are retrievable and usable for as long as needed to conduct agency business and to meet NARA-approved dispositions. Agencies must develop procedures to enable the migration of records and their associated metadata to new storage media or formats in order to avoid loss due to media decay or technology obsolescence.

(7) Execute disposition. Identify and effect the transfer of permanent records to NARA based on approved records schedules. Identify and delete temporary records that are eligible for disposal. Apply records hold or freeze on disposition when required.

(c) Backup systems. System and file backup processes and media do not provide the appropriate recordkeeping functionalities and must not be used as the agency electronic recordkeeping system.

§ 1236.22 What are the additional requirements for managing electronic mail records?

(a) Agencies must issue instructions to staff on the following retention and management requirements for electronic mail records:

(1) The names of sender and all addressee(s) and date the message was sent must be preserved for each electronic mail record in order for the context of the message to be understood. The agency may determine that other metadata is needed to meet agency business needs, e.g., receipt information.

(2) Attachments to electronic mail messages that are an integral part of the record must be preserved as part of the electronic mail record or linked to the electronic mail record with other related records.

(3) If the electronic mail system identifies users by codes or nicknames or identifies addressees only by the name of a distribution list, retain the intelligent or full names on directories or distributions lists to ensure identification of the sender and addressee(s) of messages that are records.

(4) Some e-mail systems provide calendars and task lists for users. These may meet the definition of Federal record. Calendars that meet the definition of Federal records are to be managed in accordance with the provisions of GRS 23, Item 5.

(5) Draft documents that are circulated on electronic mail systems may be records if they meet the criteria specified in 36 CFR 1222.10(b) of this subchapter.

(b) Agencies that allow employees to send and receive official electronic mail messages using a system not operated by the agency must ensure that Federal records sent or received on such systems are preserved in the appropriate agency recordkeeping system.

(c) Agencies may elect to manage electronic mail records with very short-term NARA-approved retention periods (transitory records with a very short-term retention period of 180 days or less as provided by GRS 23, Item 7, or by a NARA-approved agency records schedule) on the electronic mail system itself, without the need to copy the record to a paper or electronic recordkeeping system, provided that:

(1) Users do not delete the messages before the expiration of the NARA-approved retention period, and

(2) The system's automatic deletion rules ensure preservation of the records until the expiration of the NARA-approved retention period.

(d) Except for those electronic mail records within the scope of paragraph (c) of this section:

(1) Agencies must not use an electronic mail system to store the recordkeeping copy of electronic mail messages identified as Federal records unless that system has all of the features specified in § 1236.20(b) of this part.

(2) If the electronic mail system is not designed to be a recordkeeping system, agencies must instruct staff on how to copy Federal records from the electronic mail system to a recordkeeping system.

(e) Agencies that retain permanent electronic mail records scheduled for transfer to the National Archives must either store them in a format and on a medium that conforms to the requirements concerning transfer at 36 CFR part 1235 or maintain the ability to convert the records to the required format and medium at the time transfer is scheduled.

(f) Agencies that maintain paper recordkeeping systems must print and file their electronic mail records with the related transmission and receipt data specified by the agency's electronic mail instructions.

§ 1236.24 What are the additional requirements for managing unstructured electronic records?

(a) Agencies that manage unstructured electronic records electronically must ensure that the records are filed in a recordkeeping system that meets the requirements in § 1236.10, except that transitory e-mail may be managed in accordance with § 1236.22(c).

(b) Agencies that maintain paper files as their recordkeeping systems must establish policies and issue instructions to staff to ensure that unstructured records are printed out for filing in a way that captures any pertinent hidden text (such as comment fields) or structural relationships (e.g., among worksheets in spreadsheets or other complex documents) required to meet agency business needs.

§ 1236.26 What actions must agencies take to maintain electronic information systems?

(a) Agencies must maintain inventories of electronic information systems and review the systems periodically for conformance to established agency procedures, standards, and policies as part of the periodic reviews required by 44 U.S.C. 3506. The review should determine if the records have been properly identified and described, and if the schedule descriptions and retention periods reflect the current informational content and use. If not, agencies must submit an SF 115, Request for Records Disposition Authority, to NARA.

(b) Agencies must maintain up-to-date documentation about electronic information systems that is adequate to:

(1) Specify all technical characteristics necessary for reading and processing the records contained in the system;

(2) Identify all inputs and outputs;

(3) Define the contents of the files and records;

(4) Determine restrictions on access and use;

(5) Understand the purpose(s) and function(s) of the system;

(6) Describe update cycles or conditions and rules for adding, changing, or deleting information in the system; and

(7) Ensure the timely, authorized disposition of the records.

§ 1236.28 What additional requirements apply to the selection and maintenance of electronic records storage media for permanent records?

(a) Agencies must maintain the storage and test areas for electronic records storage media containing permanent and unscheduled records within the following temperature and relative humidity ranges:

(1) Temperature—62° to 68 °F.

(2) Relative humidity—35% to 45%.

(b) Electronic media storage libraries and test or evaluation areas that contain permanent or unscheduled records must be smoke-free.

(c) For additional guidance on the maintenance and storage of CDs and DVDS, agencies may consult the National Institute of Standards and Technology (NIST) Special Publication 500-252, Care and Handling of CDs and DVDs at http://www.itl.nist.gov/iad/894.05/papers/CDandDVDCareandHandlingGuide.pdf, contact phone number [phone].

(d) Agencies must test magnetic computer tape media no more than 6 months prior to using them to store electronic records that are unscheduled or scheduled for permanent retention. This test should verify that the magnetic computer tape media are free of permanent errors and in compliance with NIST or industry standards.

(e) Agencies must annually read a statistical sample of all magnetic computer tape media containing permanent and unscheduled records to identify any loss of data and to discover and correct the causes of data loss. In magnetic computer tape libraries with 1800 or fewer tape media, a 20% sample or a sample size of 50 media, whichever is larger, should be read. In magnetic computer tape libraries with more than 1800 media, a sample of 384 media should be read. Magnetic computer tape media with 10 or more errors should be replaced and, when possible, lost data must be restored. All other magnetic computer tape media which might have been affected by the same cause (i.e., poor quality tape, high usage, poor environment, improper handling) must be read and corrected as appropriate.

(f) Before the media are 10 years old, agencies must copy permanent or unscheduled data on magnetic records storage media onto tested and verified new electronic media.

§ 1236.30 Requirements for digitizing temporary records.

(a) If an agency intends to digitally reproduce (digitize) temporary records in order to use the digitized records in place of the source records, the agency must:

(1) Digitize the record to the standards in § 1236.32;

(2) Validate the digitization according to § 1236.34; and

(3) Manage the digital records according to the requirements in subparts A, B, and C of this part.

(b) When an agency disposes of source records, the agency must follow the requirements in § 1236.36.

§ 1236.32 Digitization standards.

When digitizing temporary records, agencies must meet the following standards:

(a) Capture all information contained in the source records;

(b) Include all the pages or parts from the source records;

(c) Ensure the agency can use the digitized records for all the purposes the source records serve, including the ability to attest to transactions and activities;

(d) Protect against unauthorized deletions, additions, or alterations to the digitized versions; and

(e) Ensure the agency can locate, retrieve, access, and use the digitized versions for the records' entire retention period.

§ 1236.34 Validating digitization.

(a) Agencies must validate that the digitized records are of suitable quality to replace source records.

(b) Agencies may establish their own validation process or use third-party processes to validate that the digitized records comply with § 1236.32. The process may be project-based or agency-wide policy.

(c) Agencies must document the validation process and retain that documentation for the life of the process or the life of any records digitized using that process, whichever is longer.

(d) NARA may review validation documentation as needed.

§ 1236.36 Disposing of source records.

(a) When an agency disposes of source records, it must have an approved agency records schedule or identify an applicable General Records Schedule.

(b) When an agency has validated that the digitized versions meet the standards in § 1236.32, the agency may destroy the source records according to General Records Schedule (GRS) 4.5 Digitizing Records.

(c) Agencies must consider any existing legal restrictions, such as a litigation hold, before destroying the source records.

(d) Agencies must manage the digitized records in the same way it would have managed the source records. Agencies must retain the digitized records for the remaining portion of any retention period established by the applicable records schedule.

(e) Agencies do not need NARA approval to destroy scheduled temporary source records they have digitized according to this part.

§ 1236.40 Scope of this subpart.

(a) This subpart establishes processes and requirements to ensure that agencies:

(1) Identify the records the agency will digitize in each project;

(2) Account for all records covered by the project, regardless of media type;

(3) Implement quality management techniques to verify equipment performance and monitor processes to detect and correct errors;

(4) Produce complete and accurate digitized records that the agency can use for all the same purposes as the source records; and

(5) Validate that the resulting digitized records meet the standards in this subpart.

(b) This subpart covers the standards and procedures agencies must apply when digitizing permanent paper records using reflective digitization techniques. Such records include most paper-based documents, regardless of size, such as modern textual documents, maps, posters, manuscripts, graphic arts prints (for example, lithographs or intaglio), drawings, bound volumes, and photographic prints. This subpart also covers any records that may be incorporated into mixed-media records.

(c) This subpart does not cover standards and procedures agencies must apply when digitizing permanent records using transmissive digitization techniques. Such records include photographic negatives, transparencies, aerial film, roll film, and micrographic and radiographic materials. In addition, this subpart does not cover digitizing records on dynamic media. Such records include motion picture film, video, and audio tapes.

(d) For guidance on digitizing out-of-scope media types or non-paper-based portions of mixed-media records, such as dynamic media, radiographic, negative or positive film, or other special media types, please contact the Records Management Policy and Standards Team by email at [email protected] or by phone at 301-837-1948.

(e) This subpart does not require that optical character recognition (OCR) be performed during digitization. However, these regulations do not prevent agencies from performing OCR to meet their business needs.

(f) This subpart does not address other applicable laws and regulations governing documents and digital files, including, but not limited to, proper handling of classified or controlled unclassified information (CUI) and compliance with 36 CFR part 1194 (section 508). Agencies should work with their legal counsel and other officials to ensure compliance with these and other applicable requirements.

(g) This subpart also does not address other business needs or legal constraints that may make it necessary for an agency to retain source records for a period of time after digitizing. Agencies should work with their legal counsel and other officials to determine whether such retention might be necessary because it relates to rights and interests, appeal rights, benefits, national security, litigation holds, or other similar reasons.

§ 1236.41 Definitions for this subpart.

In addition to the definitions contained in § 1220.18 of this subchapter and § 1236.2, the following definitions apply to this subpart:

Accuracy is the degree to which the information correctly describes the object or process being measured. It can be thought of in terms of how close a reading or average of readings is to a true or target value. Accuracy is a different measure than precision.

Adobe RGB is a red, green, blue color space developed to display on computer monitors most of the colors that CMYK color printers produce. The Adobe RGB color space is significantly larger than the sRGB color space, particularly in the cyan and green regions.

Aimpoint is a specific value assigned to a given metric to assess performance achievement.

Artifact (defect) is a general term to describe a broad range of undesirable flaws or distortions in digital reproductions produced during image capture or data processing. Some common forms of image artifacts include noise, chromatic aberration, blooming, interpolation, and imperfections created by compression.

Batch is a group of files that are created under the same conditions or are related intellectually or physically. During digitization, batches represent groups of records that are digitized and undergo QC inspection processes together.

Bit depth is the number of bits used to represent each pixel in an image. The term is sometimes used to represent bits per pixel and at other times, the total number of bits used multiplied by the number of total channels. For example, a typical color image using 8 bits per channel is often referred to as a 24-bit color image (8 bits x 3 channels). Color scanners and digital cameras typically produce 24-bit (8 bits x 3 channels) images or 36-bit (12 bits x 3 channels) capture, and high-end devices can produce 48-bit (16-bit x 3 channels) images. Bit depth is also referred to as “color depth.”

Clipping is the abrupt truncation of a signal when the signal exceeds a system's ability to differentiate signal values above or below a particular level. In the case of images, the result is that there is no differentiation of light tones when the clipping is at the high end of signal amplitude, and no differentiation of dark tones when clipping occurs at the low end of signal amplitude.

CMYK is a subtractive color model used in printing that is based on cyan (C), magenta (M), yellow (Y), and black (K). These are typically referred to as “process colors.” Cyan absorbs the red component of white light, magenta absorbs green, and yellow absorbs blue. In theory, the mix of the three colors will produce black, but black ink is also used to increase the density of black in a print.

Color accuracy is measured by computing the color difference (ΔE2000) between the digital imaging results of the standard target patches and their premeasured color values. By imaging an appropriate target and evaluating through the software, variances from known values can be determined, which is a good indicator of how accurately the system is recording color. Analytical software measures the average deviation of all color patches measured (the mean).

Color channel misregistration is the measurement of color-to-color spatial dislocation of otherwise spatially coincident color features of a digitized object.

Color management is using software, hardware, and procedures to measure and control color in an imaging system, including capture and display devices.

Color space is a specific organization of colors that supports reproducible representations of color in combination with color profiling supported by various devices. A color space can be a helpful conceptual tool for describing or understanding the color capabilities of a particular device or digital file. Examples of color spaces include Adobe RGB 1998, sRGB, ECIRGB_v2, and ProPhoto RGB.

Compression, lossless is a technique for data compression that will allow the decompressed data to be exactly the same as the original data before compression, bit-for-bit. The compression of data is achieved by coding redundant data in a more efficient manner than in the uncompressed format.

Compression, visually lossless is a form or manner of lossy compression where the data that is lost after the file is compressed and decompressed is not detectable to the human eye; the compressed data appearing identical to the uncompressed data.

Digital Image Conformance Evaluation (DICE) is the measurement and monitoring component of the Federal Agencies Digital Guidelines Initiative (FADGI) Conformance Program. The program consists of measuring ISO-compliant reference targets and using analysis software such as OpenDICE for testing and monitoring digitization programs to ensure they meet FADGI technical parameters. Agencies can access FADGI-compliant tools and resources online at http://www.digitizationguidelines.gov/guidelines/digitize-OpenDice.html.

Digitization project is any action an agency (including an agent acting on the agency's behalf, such as a contractor) takes to digitize permanent records. For example, a digitization project can range from a one-time digitization effort to a multiyear digitization process; can involve digitizing a single document into a digital records management system or digitizing boxes of records from storage facilities; or can include digitizing active records as part of an ongoing business process or digitizing inactive records for better access.

Digitized record is a digital record created by converting paper or other media formats to a digital form that is of sufficient authenticity, reliability, usability, and integrity to serve in place of the source record.

Dynamic range is the ratio between the smallest and largest possible values of a changeable quantity, frequently encountered in imaging or recorded sound. Dynamic range is another way of stating the maximum signal-to-noise ratio.

Federal Agencies Digital Guidelines Initiative (FADGI) is a collaborative effort by Federal agencies to articulate Technical Guidelines that form the basis for many of the technical parameters in this part, which equate to the FADGI three-star level. Agencies can access FADGI online at http://www.digitizationguidelines.gov/guidelines/digitize-technical.html.

Grayscale is an image type lacking any chromatic data, consisting of shades of gray ranging from white to black. Most commonly seen as having 8 bits per pixel, allowing for 256 shades or levels of intensity.

Image quality is the degree of perceived or objective measurement of a digital image's overall accuracy in faithfully reproducing an original. A digital image created to a high degree of accuracy meets or exceeds objective performance attributes (such as level of detail, tonal and color fidelity, and correct exposure), and has minimal defects (such as noise, compression artifacts, or distortion).

Lightness uniformity measures how evenly a lens records the lighting of neutral reference targets from center to edge and between points within the image.

Modulation transfer function (MTF)/spatial frequency response (SFR) is the modulation ratio between the output image and the ideal image. SFR measures the imaging system's ability to maintain contrast between progressively smaller image details. Using these two functions, a system can make an accurate determination of resolution related to the sampling frequency.

Newton's Rings are interference patterns that appear as a series of concentric, alternating light and dark rings of colored light (when imaged in a color mode). This type of interference is caused when smooth transparent surfaces come into contact with small gaps of air between the surfaces. The light waves reflect from the top and bottom surfaces of the air film formed between the surfaces, causing light rays to constructively or destructively interfere with each other. The areas where there is constructive interference will appear as light bands and the areas where there is destructive interference will appear as dark bands.

Noise is one or more undesirable image artifact(s) in a digitized record that is not part of the source material.

Pixels per inch (ppi), describes the resolution capabilities of an imaging device, such as a scanner, or the resolution of a digital image. PPI is different from dots per inch (dpi).

Posterization is an effect produced by reducing the number of tones (colors) in an image so that there is a noticeable distinction between one tone and another instead of a gradual shift between them.

Precision is the characteristic of measurement that relates to the consistency between multiple measurements, under uniform conditions, of the same item or process. As opposed to accuracy, precision does not indicate how close a measurement is to a true value.

Quantization is a lossy compression technique that involves compressing a range of values to a single quantum value, usually to reduce file size. This may result in flaws in an image, such as posterization, caused by reducing the data available in an image file to represent aspects like colors.

Raster image is a digitally encoded representation of a subject's tonal and brightness information into a bitmap. Data from digital cameras and scanning devices record light characteristics as numerical values into a grid, or raster, of picture elements (pixels).

Reference target is a chart of test patterns and patches with known standard values used to evaluate the performance of an imaging system.

Reflective digitization is a process in which an imaging system captures reflected light off of scanned objects such as bound volumes, loose pages, cartographic materials, illustrations, posters, photographic prints, or newsprint.

Reproduction scale accuracy measures the relationship between the physical size of the original object and the size in pixels per inch (PPI) of that object in the digital image.

Resolution is the level of spatial detail rendered by an imaging system as measured by MTF/SFR.

Sampling frequency measures the imaging spatial resolution and is computed as the physical pixel count or pixels per unit of measurement, such as pixels per inch (PPI). This parameter provides information about the size of the original and the data needed to determine the level of detail recorded in the file. ( See also modulation transfer function (MTF)/spatial frequency response (SFR).)

Sharpening artificially enhances details to create the illusion of greater definition. Image quality testing using the SFR quantifies the level of sharpening introduced by imaging systems or applied by users in post-processing actions.

Source record is the record from which a digitized version or digitized record is created. The source record should be the record copy that was used in the course of agency business.

Spatial resolution determines the amount (for example, quantity, PPI, megapixels) of data in a raster image file in terms of the number of picture elements or pixels per unit of measurement, but it does not define or guarantee the quality of the information. Spatial resolution defines how finely or widely spaced the individual pixels are from each other. The actual rendition of fine detail is more dependent on the SFR of the scanner or digital camera.

sRGB is a standard RGB color space created by HP and Microsoft for use on monitors, printers, and the internet. sRGB uses the ITU-R BT.709-5 primaries that are also used in studio monitors and HDTV, and a transfer function (gamma correction) typical of CRTs (cathode ray tube TVs and computer monitors), all of which permits sRGB to be directly displayed on typical monitors. The sRGB gamma is not represented by a single numerical value. The overall gamma is approximately 2.2, consisting of a linear (gamma 1.0) section near black, and a non-linear section elsewhere involving a 2.4 exponent and a gamma changing from 1.0 through about 2.3.

Tolerance is the allowable deviation from a specified value.

Tone response or optoelectronic conversion function (OECF) is a measure of how accurately the digital imaging system converts light levels into digital pixels.

Transmissive digitization is a process in which the system transmits light through a photographic slide or negative.

White balance error is a measurement of the digital file's color neutrality. The definition of “neutral” is not universal: RGB workflows that use digital count values encode neutral as defined by the International Color Consortium (ICC) color space chosen. L*a*b* workflows define neutral as 0 on the a* axis and b* axis, with the lightness recorded from 0-100 on the L* axis.

§ 1236.42 Records management requirements.

(a) Before starting a digitization project, agencies must establish intellectual control of the records that will be digitized. Intellectual control means having the information necessary to identify and understand the content and context of the records. One traditional records management technique to establish intellectual control is the creation of an inventory. The inventory must identify whether the records are complete, if there are any gaps in coverage or missing records, the presence of any mixed-media records, the disposition schedule under which the records fall, the date range when the records were created, any access or use restrictions that apply to the records, and the records' storage location.

(b) Agencies must identify any relationships between the source records in order to retain these relationships between the digitized versions. For example, are there case files that are associated by case number? Does a folder contain multiple documents that are stapled together? Are there digital components of a mixed-media file stored on removable media (DVD or USB drives)? What is the relationship of the folder to other folders in a box? Any relationships must be captured as part of the digitization process:

(1) Through metadata (See § 1236.54 for metadata requirements);

(2) By organizing the folder structure of a file system;

(3) By using file formats that allow for multi-page files, such as PDF or TIFF; or

(4) Through a combination of these approaches.

(c) In addition, the inventory can be used to identify all the elements of physical control needed for the records to be digitized. Physical control includes understanding the physical characteristics of source records. Physical characteristics determine a project's scope, and the image capture techniques and equipment to be used. For example, the type of paper, the type of printing, or the size of the records can impact what methods and equipment are used to digitize records.

(d) There are additional considerations for managing the source records during the digitization process:

(1) Ensure there are appropriate safeguards for the source records to prevent their loss or damage.

(2) Restrict access to source records while they are being digitized to minimize the risk of unauthorized additions, deletions, or alterations.

(3) Ensure there is a process to identify and document gaps in coverage or missing records.

(e) Agencies must ensure that records are free from unauthorized alteration, destruction, or deletion by complying with the mechanisms and controls specified in §§ 1236.10 and 1236.20:

(1) The agency may generate checksums using the SHA-256 hash algorithm and record them as technical metadata in a recordkeeping system for each image file when digitization is complete and the agency determines that the records are no longer in active use and the metadata are no longer subject to any changes that may result from ongoing business use. Use the checksums to monitor the digitized records for corruption or alteration and capture them as metadata as required in § 1236.54; or

(2) The agency may perform file integrity monitoring or file comparison audits.

(f) If there are born-digital records that are part of the record series within the project, follow the instructions for managing mixed-media records in § 1236.52.

§ 1236.44 Documenting digitization projects.

Agencies must create digital documentation when digitizing permanent source records. The agency must retain this documentation alongside the digitized records until the digitized records have been transferred to NARA and NARA has notified the agency that the accessioning process is complete. The agency must dispose of the documentation in accordance with an appropriate General Records Schedule (GRS) or agency records schedule. The required documentation will help the agency populate the Transfer Request instrument (TR) in NARA's Electronic Records Archives (ERA). The following documents are required:

(a) A defined project plan that identifies:

(1) Record series or file units to be digitized;

(2) Method that will be used to name digitized records;

(3) Estimated date range of the source records;

(4) Missing pages;

(5) Gaps or missing records in the series. Depending on the type of gap or missing records, indicate if there will be charge-out cards for skipped or missing records that will be inter-filed if they are transferred at a later date;

(6) Estimated volume, media types, dimensions, physical characteristics, and condition of the source records;

(7) Equipment and software used to digitize records;

(8) Estimated file storage requirements for the digitized records. The file storage needs may affect project decisions, such as compression and file format;

(9) Any access or use restrictions that apply to the records;

(10) Method used to capture the relationships that exist between source records once they are digitized; and

(11) Any metadata element labels that differ from those specified in § 1236.54.

(b) Any information needed to associate the digitized records to the source records' agency records schedule(s) including the item numbers;

(c) Any related finding aids, indexes, inventories, logs, registers, or metadata schemas the agency uses to manage the records that can serve as sources for the metadata required in § 1236.54.

(d) A quality management (QM) plan that ensures the project meets the quality assurance (QA) objectives and quality control (QC) inspection procedures.

(1) The quality management plan must include the policies, functions, roles, responsibilities, requirements, and objectives for the project.

(2) The quality assurance component of the QM plan must include documentation of:

(i) Image quality performance parameters selected to capture the information present in the source records;

(ii) Equipment and device acceptance testing methods and results;

(iii) Design reviews to evaluate if digitization workflows meet the requirements; and

(iv) Training conducted.

(3) The quality control component of the QM plan must document:

(i) The procedures used to inspect image quality;

(ii) The procedures used to inspect metadata quality;

(iii) The corrective actions taken to mitigate deviations throughout all phases of the project; and

(iv) The procedures used to verify that digitized records conform to the requirements.

§ 1236.46 Quality management requirements.

(a) Quality assurance (QA) requirements. The agency must meet the image quality performance parameters specified in § 1236.50 by verifying how well the equipment meets the aim points and tolerances of the parameters. The agency cannot rely solely on equipment specifications, such as scanner ppi settings or camera sensor megapixels, to ensure digital image quality.

(1) The agency must use QA processes to:

(i) Quantify scanner or camera performance before selecting the equipment by scanning a reference target and measuring the results with analytical software to determine if the equipment meets the technical parameters.

(ii) Evaluate internal or external vendor imaging systems against image quality performance parameters;

(iii) Monitor equipment performance by quantifying scanner or camera performance during digitization; and

(iv) Verify that resulting digital files meet project specifications.

(b) Quality Control (QC) requirements. The agency must implement QC inspection and monitoring processes to ensure that images meet the digitization image quality parameters in § 1236.50.

(1) The Federal Agencies Digital Guidelines Initiative (FADGI) Digital Image Conformance Evaluation program (DICE) is a QC inspection and monitoring process that uses image targets and analysis software to verify compliance. Applied properly, this methodology will ensure agencies meet the requirements in § 1236.50.

(2) If the agency does not adopt the FADGI Conformance Evaluation program, it must document both the procedures used and how it verified conformance to the quality parameters.

(c) Quality Control (QC) testing and analysis. During the digitization process, the agency must perform QC testing and analysis to identify malfunctioning or improperly configured digitization equipment, improper software application settings, incorrect metadata capture, or human error, and take corrective actions. It must:

(1) Implement an image quality analysis process and use reference targets to verify that digitization devices conform to imaging parameters in this subpart;

(2) Replace reference targets as they fade or accumulate dirt, scratches, and other surface marks that reduce their usability;

(3) Regularly test equipment to ensure scanners and digital cameras/copy systems are performing optimally. It must:

(i) Scan a reference target containing a grayscale, color chart, and accurate dimensional scale at the beginning of each workday;

(ii) Use image quality analysis software to verify that the performance evaluation specifications are being met; and

(iii) Perform additional tests when problems are detected.

(4) Test equipment with the specific software/device driver combination(s), and re-test after any changes to the workflow; and

(5) Ensure that equipment operation, settings, and image processing actions are the same as those used to evaluate the test target. Turn off auto correction settings in the capture equipment such as “auto exposure” that may cause non-conformance of the target evaluation or the resulting image files.

(d) Quality control inspection. (1) The agency must perform QC inspections of the digital records for compliance with the technical parameters and criteria specified in this subpart. The inspection must ensure 100% of the image files:

(i) Can open and be displayed;

(ii) Are encoded with a compression type and in a format specified in § 1236.48; and

(iii) Have the resolution, color mode, bit depth, and color profile specified in § 1236.50.

(2) The agency must perform a visual inspection using a statistically valid technique:

(i) The agency may visually inspect a random sample of a minimum of ten digital records or 10% of each batch of digital records, whichever is larger; or

(ii) The agency may employ a statistically valid sampling plan to verify that the image quality, file quality, metadata quality, and completeness requirements have been met. Agencies that employ their own sampling technique must include documentation of the method used, as specified in § 1236.44(d)(3)(i).

(3) Visual inspection must be conducted using a calibrated graphics workstation and using a monitor set to 100% magnification to check the following image quality characteristics:

(i) Image tone, brightness, contrast, and color accuracy match the specifications in § 1236.50;

(ii) Images are free from clipping (missing detail lost in highlights or shadows);

(iii) Images are free from color channel misregistration, or quantization errors;

(iv) Images are free of any image artifacts that compromise the informational content of the record, such as dust, Newton's rings, missing pixels, scan lines, drop-outs, flare, or over-sharpening; and

(v) Images are not improperly cropped, have the expected dimensions and orientation (landscape/horizontal or portrait/vertical), and images are not flipped, inverted, or skewed.

(e) Corrective measures. If the inspection reveals errors, perform the following steps until there is a 100% success rate for the sample set:

(1) If 1% or more of examined records fail to meet any of the criteria in § 1236.50, determine the source and scope of any errors, correct or re-digitize affected records, and reinspect the images by following the requirements in paragraph (d) of this section;

(2) If less than 1% of examined records fail to meet any of the criteria in § 1236.50, determine the source and scope of any errors and correct or re-digitize the affected records.

(f) Inspection for other quality aspects. The agency must inspect the resulting files to verify that they meet the metadata and records completeness requirements:

(1) Metadata quality. The agency must evaluate the accuracy of metadata. This may be done using automated techniques if appropriate. Otherwise, the QC inspections must be done manually. These inspections must ensure that:

(i) Files are named according to project specifications; and

(ii) Correct administrative, descriptive, and technical metadata are captured in a recordkeeping system and in image files.

(2) Records completeness. The agency must employ automated and visual inspection processes to verify the completeness and accuracy of digitization:

(i) Verify that all records have been accounted for by referring to box lists, folder title lists, or other inventories;

(ii) Compare source records with their digitized versions to verify that 100% of the informational content has been captured;

(iii) Compare source records with their digitized versions to verify the digitized records are in the same order;

(iv) Examine records for related envelopes, notes, or other forms of media to verify that all sources of record information have been digitized;

(v) Verify that any mixed-media records that cannot be digitized are associated with the digitized records using the “Relation” metadata elements in § 1236.54(c); and

(vi) Confirm that missing pages or images have been noted in the project documentation.

§ 1236.48 File format requirements.

(a) The agency must encode, retain, and transfer digitized records in one of the following file formats, either uncompressed or using one of the specified compression codecs in tables 1 and 2 to this section.

(1) Agencies that combine multiple uncompressed TIFF images into PDF/A files using JPEG2000 compression must perform the quality inspection step specified in 1236.46(d) against the resulting PDF/A files.

(2) When using JPEG 2000 visually lossless compression, agencies must determine the amount of compression to apply, not to exceed 20:1, by performing tests and visually evaluating for compression artifacts that obscure or alter the information content.

(b) The agency must encode, retain, and transfer digitized permanent paper records in one of the following file formats, either uncompressed or with one of the compression codecs specified in table 1 to this paragraph (b).

Table 1 to Paragraph ( b )—File Format Requirements for Digitized Permanent Paper Records Table

Format name and version

Acceptable compression codecs

TIFF 6.0

Uncompressed, Deflate (ZIP).

JPEG2000 part 1 (ISO/IEC 15444-1:2019)

JPEG 2000 part 1 core coding system lossless compression. Agencies may use up to 20:1 visually lossless compression.

Portable network graphics 1.2 (PNG)

Deflate (ZIP).

PDF/A (Select any version of PDF/A that meets project requirements. However, do not use the attachments feature in PDF/A-3 or PDF/A-4

Deflate (ZIP), JPEG 2000 part 1 core coding system lossless compression. Agencies may use up to 20:1 visually lossless compression.

(c) The agency must encode, retain, and transfer digitized photographic print records in one of the following file formats, either uncompressed or with one of the compression codecs specified in the table 2 to this paragraph (c).

(1) For a series of predominantly textual records with interspersed photographic prints, use the formats in table 1 to paragraph (b) of this section for paper records. All photographic prints must be digitized according to the standards in § 1236.50.

(2) For a series of predominantly printed photographs, including those with paper records interspersed, use the file formats in table 2 to this paragraph (c) for photographic print records.

(3) However, the agency must not transcode, or interpolate (upsample) files anywhere in the workflow.

Table 2 to Paragraph ( c )—File Format Requirements for Digitized Permanent Photographic Print Records Table

Format name and version

Acceptable compression codecs

TIFF 6.0

Uncompressed, Deflate (ZIP).

JPEG2000 part 1 (ISO/IEC 15444-1:2019)

JPEG 2000 part 1 core coding system lossless compression. Agencies may use up to 20:1 visually lossless compression.

Portable network graphics 1.2 (PNG)

Deflate (ZIP).

§ 1236.50 Requirements for digitizing permanent paper and photographic print records.

(a) Overview. This section describes the minimum requirements appropriate for digitizing paper records. Depending on the physical characteristics of the source records, the agency must select the applicable specifications described in either table 1 to paragraph (d) of this section for modern textual paper records or the table 2 to paragraph (e) of this section for photographic prints and paper records with fine details. Agencies must implement appropriate equipment, lighting, special handling, or imaging methods to ensure the capture of all information. Agencies may exceed these requirements, if necessary, to capture fine detail or to meet their own business needs.

(b) Image quality parameters. The performance parameters are based on FADGI three-star aim points and tolerance ranges.

(c) Equipment requirements. The equipment used to digitize Federal records must be appropriate for the media type, and capable of achieving documented project objectives without damaging the source records.

(d) Requirements for digitizing modern textual paper records. For these records, produce image files at a minimum of 300 ppi sized to the source document.

(1) Records suitable for the specifications in table 1 to this paragraph (d) for modern textual paper records are modern textual documents with a well-defined printed type (such as typeset, typed, laser-printed), and with moderate to high contrast between the ink of the text and the paper background. Performance metric values in table 1 for modern textual paper records conform to the FADGI “Documents (Unbound): Modern Textual Records” category, and are appropriate when source records do not have visible content with L* values darker than 20. Neutral reference patches on the evaluation test target with L* less than 20 are not used for analysis.

(2) For other paper records such as manuscripts, illustrations, graphics, and documents with poor legibility or diffuse characters (such as carbon copies or Thermofax) that have visible content with L* values darker than 20, agencies must evaluate neutral reference patches on the evaluation test target with L* greater than 20. (These values equate to FADGI three-star for “Documents (Unbound): General Collections”).

(3) The agency must digitize in an acceptable RGB color mode if records contain color or other characteristics that are necessary to interpret the information of the source record, or that would be lost when digitizing using grayscale gamma 2.2.

(4) At a minimum, the agency must digitize the paper records covered by this paragraph to the following parameters:

Table 1 to Paragraph ( d )—Requirements for Digitizing Permanent, Modern Textual Paper Records Table

Digital file specifications

Attributes

Color mode

color or grayscale.

Bit depth

8 or 16.

Color space

gray gamma 2.2, AdobeRGB1998, sRGB, ProPhoto RGB, ECIRGBv2.

Resolution (Sampling Frequency) (Units are Pixels Per Inch/ppi minus Reproduction Scale Accuracy)

≥294 ppi (300 ppi—2%).

Measurement parameters

Performance metric values

Difference from aim (applies to 20 ≤ L* ≤ 100).

Tone Response (OECF) L* (Units Colorimetric ΔL*) gray patches that meet the measurement parameters

± 5.

White Balance (Units Colorimetric ΔE(a*b*)) gray patches that meet the measurement parameters

≤4%.

Lightness Uniformity (Units Colorimetric—Standard Deviation Divided by Mean L*)

≤3%.

Average Color Accuracy (Units Colorimetric—Mean ΔE 2000—for patches meeting the measurement parameters)

≤ 3.5.

Color Accuracy 90th Percentile (Units Colorimetric—2.5 times average deviation for patches meeting the measurement parameters)

≤ 8.75.

Color Channel Misregistration (Units Pixels)

< 0.5 pixel.

SFR 10 (Sampling Efficiency) (Measurement is a Ratio %)

>80%.

MTF50 (50% SFR) (Percentage of Half Sampling Frequency) [Lower, Upper]

Percentage of half sampling frequency: [>40%, <75%].

Reproduction Scale Accuracy (Units % Difference from Header PPI)

<±2%.

Sharpening (Units Max Modulation)

< 1.1.

Noise (Upper Limit) (Units Std Dev of L*)

≤ 2.

Noise (Warning Limit) (Units Std Dev of L*)

≥.25.

(e) Requirements for digitizing photographic prints and paper records that have fine details. Records that have fine detail, require a high degree of color accuracy, or have other unique characteristics, must be captured using the specifications in table 2 to this paragraph (e) for photographic prints and paper records with fine details. For these records, produce image files (as described table 2) at a minimum of 400 ppi sized to the source document (these performance values equate to FADGI three-star category “Prints and Photographs”). It may be necessary to apply a higher resolution than the minimum for some records that have fine detail.

(1) These specifications apply to records such as photographic prints, graphic-arts prints (for example, lithographs or intaglio), drawings, embossed seals, and records that have information that cannot be captured by the parameters in table 1 to paragraph (d) of this section for modern textual paper records.

(i) For records in which the smallest significant detail is 1.0 mm or smaller, such as aerial photographs and topographic maps (which require a high degree of enlargement and precision to ensure the dimensional accuracy of the scans), the agency must increase the resolution to capture all the information in the source record.

(ii) For many imaging devices, increasing the ppi settings may not increase the actual resolution level or capture the desired detail. The equipment for digitizing records with fine detail must be capable of meeting the higher quality parameters. It may be necessary to exceed the parameters in table 2 to this paragraph (e) to capture all the information inherent in the records.

(2) The agency must digitize photographic prints, including monochrome and black and white, using a color mode.

(3) The agency must digitize in an acceptable color mode if records contain color or other characteristics that are necessary to interpret the information of the source record, or that would be lost when digitizing using grayscale gamma 2.2.

(4) At a minimum, agencies must digitize all records covered by this paragraph to the following parameters:

Table 2 to Paragraph ( e )—Requirements for Digitizing Permanent, Photographic Print Records and Paper Records That Have Fine Details

Digital file specifications

Attributes

Color mode

color or grayscale.

Bit depth

8 or 16.

Color space

Gray gamma 2.2, AdobeRGB1998, ProPhoto RGB, ECIRGBv2.

Resolution (Sampling Frequency) (Units are Pixels Per Inch/ppi minus Reproduction Scale Accuracy)

≥392 ppi (400 ppi—2%).

Measurement parameters

Performance metric values

Tone Response (OECF) L* (Units Colorimetric ΔL2000*) for any given gray patch

± 4.

White Balance (Units Colorimetric ΔE(a*b*)) for any given gray patch

≤4.

Lightness Uniformity (Units Colorimetric − Standard Deviation Divided by Mean)

<3%.

Average Color Accuracy(Units Colorimetric—Mean ΔE 2000—average deviation of all patches)

<3.5.

Color Accuracy 90th Percentile (Units Colorimetric—2.5 times average deviation of all patches)

<8.75.

Color Channel Misregistration (Units Pixels)

<0.5 pixel.

SFR10 (Sampling Efficiency) (Measurement is a Ratio %)

80%.

SFR50 (50% SFR) (Units Percentage of Half Sampling Frequency) [Lower, Upper]

Percentage of half sampling frequency: [>40%, <75%].

Reproduction Scale Accuracy (Units % Difference from Header PPI)

<± 2%.

Sharpening (Units Max Modulation)

<1.1.

Noise (Upper Limit) (Units Std Dev of L*)

<2.

Noise (Lower Limit) (Units Std Dev of L*)—A warning should be raised if the image doesn't meet this criteria

≥.25.

§ 1236.52 Requirements for digitizing permanent mixed-media records.

Mixed-media files are records that belong together or relate to a common topic and are stored on more than one media type. Mixed-media files result from the processes agencies use to create, maintain, and use records. For example, a case file may include paper records, online digital records, and digital records on storage media.

(a) For any non-paper media, agencies must analyze the contents to determine whether any files are records.

(1) If the media contains records that are temporary, manage them according to their appropriate GRS or agency-specific records authority.

(2) If the media contains records that are permanent, but not part of the digitized record series, locate their disposition schedule and capture them in a digital information system that complies with the requirements in § 1222.26 of this subchapter and §§ 1236.10 through 1236.14.

(3) If the media contains born-digital components of mixed-media files that are related to the digitized records series, capture the born-digital records in a recordkeeping system in accordance with § 1222.26 of this subchapter and associate the born-digital records with any related records once they are digitized using the “Relation” metadata elements in § 1236.54.

(4) If they are permanent records stored on a media type that is out of scope for this subpart, document this information according to the instructions in § 1236.44. Agencies must maintain the association between records using the “Relation” metadata elements specified in § 1236.54.

(b) Contact the Records Management Policy and Standards Team at [email protected] for guidance on what to do with types of media in a mixed-media file that are outside the scope of this subpart, such as dynamic media, x-rays, negative or positive film, or other special media types.

§ 1236.54 Metadata requirements.

(a) General. To ensure that intellectual and physical control of the digital records can be maintained, this regulation specifies metadata elements that must be captured in a recordkeeping system, or embedded in each file, or both captured in a recordkeeping system and embedded in each file. Ensure that the metadata remains accurate and consistent regardless of where it is stored.

(1) If using metadata to capture relationships between source records as required in § 1236.42(b), agencies must use the “Relation” metadata elements in table 1 to paragraph (c)(1) of this section for basic administrative metadata.

(2) If using different metadata labels from the ones required in this section, agencies must document the labels that the agency uses and note this in the Details section of the ERA (Electronic Records Archive) Transfer Request (TR).

(3) Determine the appropriate level to be used as the source of descriptive metadata. Depending on the agency's existing recordkeeping practices and level of intellectual control, use information from the project, record series, file unit, or item level as the source for administrative, technical, and descriptive metadata fields. If the components of a record have not been individually indexed with unique descriptions, apply the series or file unit-level descriptions to all of the image files within that grouping. If the components of the record do not have individual titles, the agency must apply the item Record IDs instead.

(4) Include additional metadata if it is captured. If other metadata elements are provided in addition to the metadata requirements in this subpart, NARA will accept that metadata as part of the transfer process.

(b) Metadata capture requirements. Agencies must:

(1) Capture the metadata specified by paragraphs (c) through (e) of this section at the file or item level as part of the digitization project;

(2) Create file names and record IDs that are unique to each image file;

(3) Embed the metadata specified by paragraph (c) of this section in each image file, capture and maintain it in a recordkeeping system, associate it with the records it describes, and keep it consistent and accurate in both places;

(4) Ensure that scanning equipment embeds the system-generated technical metadata specified by table 4 to paragraph (e)(1) of this section for format technical metadata and table 5 to paragraph (e)(2) of this section for processing technical metadata in each image file, and ensure that image processing does not alter or delete it; and

(5) Transfer metadata to NARA in CSV format.

(c) Administrative metadata. (1) Capture in a recordkeeping system and embed in each image file the following administrative metadata:

Table 1 to Paragraph ( c )(1)—Basic Administrative Metadata

Metadata label

Description

Requirement level

Identifier: File Name

The complete name of the computer file, including its extension

Mandatory (file names are an inherent attribute of each file so there is no need to embed them as an element of metadata).

Identifier: Record ID

The unique identifier assigned by an agency or a records management system. 36 CFR 1236.20(b)(1) requires that agencies assign unique identifiers to each record

Mandatory.

Identifier: Records Schedule Item #

The number assigned to the agency records schedule or GRS item to which the record belongs

Mandatory.

Relation: Has Part

A related record that is either physically or logically required in order to form a complete record. Mixed-media files that contain records on multiple media types must use this element to identify all components

Mandatory if a record includes multiple parts, such as the component parts of a case file or mixed-media file.

Relation: Is Part Of

A related record or file in which the described record is physically or logically included. Use this element to indicate that a record is a component of a mixed-media file

Mandatory if file is a component of a multi-part record.

(2) Capture in a recordkeeping system and embed in each file any of the following access and use restrictions the metadata inherited from the source records:

Table 2 to Paragraph ( c )(2)—Access and Use Restrictions Administrative Metadata

Metadata label

Required fields

Description

Requirement level

Access Restrictions

Access Restriction Status

Indicate whether or not there are access restrictions on the record

Mandatory.

Specific Access Restriction

Specific access restrictions on the record, based on national security considerations, donor restrictions, court orders, and other statutory or regulatory provisions, including Privacy Act and Freedom of Information Act (FOIA) exemptions

Mandatory if access restriction exists

Use Restrictions

Use Restriction Status

Indicate whether or not there are use restrictions on the record

Mandatory.

Specific Use Restriction

The type of use restrictions on the record, based on copyright, trademark, service mark, donor, or statutory provisions

Mandatory if use restriction exists.

Rights: Rights Holder

A person or organization owning or managing intellectual property rights relating to the record

Mandatory if there is a rights holder.

(d) Descriptive metadata. Capture the following descriptive metadata from source records at the lowest level needed to support access and preservation and to maintain contextual information. Depending on the agency's existing recordkeeping practices and level of intellectual control, it may use information from the project level, record series, file unit, or item, as the source for descriptive metadata. If the components of a record have not been individually indexed with unique descriptions, apply the series or file unit-level descriptions to all of the image files within that grouping. If source records share a common material type or dimensions, auto-populate the source type and source dimension metadata. If the components of the record do not have individual titles, the agency must apply the item Record IDs instead. Capture the metadata in a recordkeeping system for each image file:

Table 3 to Paragraph ( d )—Descriptive Metadata Table

Metadata label

Description

Requirement level

Title

A name given to the source record. If a name does not exist, the mandatory metadata element Identifier: Record ID serves as the title for the record

Mandatory.

Description

A narrative description of the content of the record, including abstracts of documents

Mandatory.

Creator

The agent (person, agency, other organization, etc.) primarily responsible for creating the source record

Mandatory.

Date: Creation Date

The date or date range indicating when the source record met the definition of a Federal record

Mandatory.

Source Type

The medium of the source record that was scanned to create a digital still image

Mandatory.

Source Dimensions

The dimensions of the source record (including unit of measure)

Mandatory.

(e) Technical metadata. (1) Ensure that the following values are embedded in each image file and that image processing does not delete or alter them:

Table 4 to Paragraph ( e )(1)—Technical Metadata—Image Table

Metadata label

Definition

Requirement level

Date Time Created

The date or date-and-time the digital image was created

Mandatory.

Image Width

The width of the digital image, i.e., horizontal or X dimension, in pixels

Mandatory.

Image Height

The height of the digital image, i.e., vertical or Y dimension, in pixels

Mandatory.

Color Space

The name of the International Color Consortium (ICC) profile used

Mandatory.

Bits Per Sample

Number of bits per component

Mandatory.

Samples Per Pixel

The number of components per pixel. Usually, 1 for grayscale images and 3 for RGB images

Mandatory.

(2) Ensure that the following process metadata elements are recorded for each image file:

Table 5 to Paragraph ( e )(2)—Technical Metadata—Process Table

Metadata label

Definition

Requirement level

Scanner Make and Model

The manufacturer and model of the scanner used to create the image

Mandatory if using a scanner.

Digital Camera Make and Model

The manufacturer and model of the digital camera used to create the image

Mandatory if using a digital camera.

Software Name and Version

The name and version of the software used to capture the image

Mandatory if using scanning software.

(3) Capture the following technical metadata in a recordkeeping system for each image file, and use them to monitor digital records for corruption or alteration:

Table 6 to Paragraph ( e )(3)—Technical Metadata—Checksum Table

Fixity metadata label

Description

Requirement level

Message Digest Algorithm

The specific algorithm used to construct the message digest for the digital object or bitstream

Mandatory if using checksums as described in § 1236.42(e)(1).

Message Digest (checksum)

The output of Message Digest Algorithm

Mandatory if using checksums as described in § 1236.42(e)(1).

§ 1236.56 Validating digitized records and disposition authorities.

(a) When a digitization project is complete, the agency must validate that the digitized versions meet the standards in this subpart.

(b) Separate staff must conduct the validation, independent from the staff that performed the digitization QC inspections described in § 1236.46.

(c) Agencies must verify that:

(1) All records identified in the project's scope have either been digitized or have been identified in project documentation as missing or incomplete records (and the agency must note this information in the Details section of the ERA TR when transferring the records);

(2) All required metadata are accurate, complete, and correctly labeled;

(3) All image technical attributes specified in § 1236.50 have been met;

(4) All image files are legible and all physical characteristics necessary to understand and use the records have been captured;

(5) Mixed-media files are digitized appropriately for the material type, or if mixed-media components are retained in their original format, they are associated with digitized components through metadata, per the requirements specified in § 1236.54(c); and

(6) Project documentation has been created according to § 1236.44.

(d) Once validated, the digitized records are permanent records.

(e) After validating, the agency must determine whether the agency has any reasons for retaining the source records for a period of time once digitized, in keeping with § 1236.40(g).

(f) Unless source records will be retained for reasons identified in § 1236.40(g), the agency must dispose of the source records in accordance with an agency records schedule or GRS that addresses disposition after digitization.

(g) Agencies cannot use the GRS to dispose of source records if the digitized records do not meet the requirements in this subpart. In such cases, agencies should contact the Records Management Policy and Standards Team at [email protected] to determine what steps they must take.

(h) Agencies must transfer the digitized records to NARA according to the approved disposition authority and include the transfer metadata as described in § 1236.58.

(i) Agencies must retain the project documentation described in § 1236.44 until the National Archives confirms receipt of the records and legal custody of the records has been transferred.

(j) Agencies must transfer the administrative, technical, and descriptive metadata captured during the digitization project as CSV files, as described in § 1236.54(b)(6), with the resulting digitized records.

§ 1236.58 Transfer metadata.

When agencies transfer permanent records to the National Archives' legal and physical custody, the agency must provide transfer metadata to NARA. The transfer metadata must be entered into the Electronic Records Archives (ERA) when the Transfer Request (TR) is created to begin transferring the records. Each transfer of digital records must include the following metadata that applies to the transfer as a whole:

Table 1 to § 1236.58—Transfer Metadata Table

Metadata label

Required fields

Description

Requirement level

Transfer Request Number

Transfer Request Number

The number automatically generated when a Transfer Request is created

Mandatory.

Transfer Title

Transfer Title

The name assigned to the collection, set or series of records the agency is transferring to NARA

Mandatory.

Dates

Inclusive Start Date

The beginning date on which the record group, collection, series, or set the agency is transferring to NARA was created, maintained, or accumulated by the creator

Mandatory.

Inclusive End Date

The last date on which the record group, collection, series, or set the agency is transferring to NARA was created, maintained, or accumulated by the creator

Mandatory.

Creating Organization

Creating Organization

The name of the organization responsible for creating, accumulating, or maintaining the collection, series, or set when in working (primary) use

Mandatory.

Record Group Number

Parent Record Group Number

The unique number assigned to a record group

Mandatory.

General Records Type

General Records Type

The general form of the records set, series, or collection the agency is transferring, including but not limited to: architectural and engineering drawings, artifacts, data files, maps and charts, moving images, photographs, and other graphic materials, sound recordings, textual records, or web pages

Mandatory.

Access Restrictions

Access Restriction Status

Indicate whether or not there are access restrictions on the set, collection, or series of records the agency is transferring to NARA

Mandatory.

Specific Access Restriction

Specific access restrictions on the set, collection, or series of records, including but not limited to: restrictions based on national security considerations, donor restrictions, court orders, and other statutory or regulatory provisions, including Privacy Act and Freedom of Information Act (FOIA) exemptions

Mandatory if access restriction exists.

Use Restrictions

Use Restriction Status

Indicate whether or not there are use restrictions on the set, collection, or series of records transferring to NARA

Mandatory.

Specific Use Restriction

The type of use restrictions on the set, collection, or series of records, including but not limited to restrictions based on: copyright, trademark, service mark, donor, or statutory provisions, including Privacy Act and Freedom of Information Act (FOIA) exemptions

Mandatory if use restriction exists.

Records Schedule Number

Records Schedule Number

The number NARA assigned to the agency records schedule that applies to all the records in the collection, series, or set transferring

Mandatory.