# WRLC Storage Journal Deduplication Procedure

WRLC Storage Journal Deduplication Procedure

1\. Rationale

The WRLC [Shared Copy Policy](http://www.libraries.wrlc.org/storage/sharedcopypolicy.html) mandates that a single copy of any given periodical volume may be placed in the WRLC Storage facility. The policy is intended to make use of warehouse space more efficient by not storing multiple copies of the same periodical volume. Once a copy is in storage, any other copies of the same volume may not be stored; they must be kept at the owning library or discarded. The WRLC Library Directors Council has directed WRLC to perform the work of identifying which periodical volumes are duplicates as a centralized process at the WRLC facility.

2\. Identifying duplicate volumes

A duplicate periodical volume is one which contains precisely the same content as a volume already in storage. All of the content of a volume sent for processing must be included in the stored copy for that volume to be considered a duplicate. Where a volume in storage has some of the content of an incoming volume, but not all, the incoming volume is not considered a duplicate.

Examples:

<table border="1" cellpadding="0" cellspacing="0" class="MsoNormalTable" id="bkmrk-incoming-volume-stor" style="border-collapse: collapse; border: none; mso-border-alt: solid black .5pt; mso-yfti-tbllook: 1184; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-border-insideh: .5pt solid black; mso-border-insidev: .5pt solid black;"><tbody><tr style="mso-yfti-irow: 0; mso-yfti-firstrow: yes;"><td style="width: 159.6pt; border: solid black 1.0pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Incoming volume

</td><td style="width: 159.6pt; border: solid black 1.0pt; border-left: none; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Stored volume

</td><td style="width: 159.6pt; border: solid black 1.0pt; border-left: none; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213"></td></tr><tr style="mso-yfti-irow: 1;"><td style="width: 159.6pt; border: solid black 1.0pt; border-top: none; mso-border-top-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65<span style="mso-spacerun: yes;"> </span>1970

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65<span style="mso-spacerun: yes;"> </span>1970

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Duplicate; same content

</td></tr><tr style="mso-yfti-irow: 2;"><td style="width: 159.6pt; border: solid black 1.0pt; border-top: none; mso-border-top-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65 no.1-2<span style="mso-spacerun: yes;"> </span>1970

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65 no.1-2<span style="mso-spacerun: yes;"> </span>1970

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Duplicate; same content

</td></tr><tr style="mso-yfti-irow: 3;"><td style="width: 159.6pt; border: solid black 1.0pt; border-top: none; mso-border-top-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65<span style="mso-spacerun: yes;"> </span>1970

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.64-65<span style="mso-spacerun: yes;"> </span>1969-70

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Duplicate; all content in stored copy

</td></tr><tr style="mso-yfti-irow: 4;"><td style="width: 159.6pt; border: solid black 1.0pt; border-top: none; mso-border-top-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65-66<span style="mso-spacerun: yes;"> </span>1970-71

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65<span style="mso-spacerun: yes;"> </span>1970

v.66<span style="mso-spacerun: yes;"> </span>1971

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Duplicate; all content in stored copies

</td></tr><tr style="mso-yfti-irow: 5;"><td style="width: 159.6pt; border: solid black 1.0pt; border-top: none; mso-border-top-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65<span style="mso-spacerun: yes;"> </span>1970

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65 no.1-2 (1970)

v.65 no. 2-4 (1970)

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Possible duplicate; set aside for review (no.1-4 may be complete, but don’t know how many numbers are in a volume)

</td></tr><tr style="mso-yfti-irow: 6;"><td style="width: 159.6pt; border: solid black 1.0pt; border-top: none; mso-border-top-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65 1970

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65 no.2-4 (1970)

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Not a duplicate; stored volume is incomplete

</td></tr><tr style="mso-yfti-irow: 7;"><td style="width: 159.6pt; border: solid black 1.0pt; border-top: none; mso-border-top-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65 1970

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65<span style="mso-spacerun: yes;"> </span>1970<span style="mso-spacerun: yes;"> </span>(inc.)

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Not a duplicate; stored volume is incomplete

</td></tr><tr style="mso-yfti-irow: 8;"><td style="width: 159.6pt; border: solid black 1.0pt; border-top: none; mso-border-top-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65-66<span style="mso-spacerun: yes;"> </span>1970-71

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65<span style="mso-spacerun: yes;"> </span>1970

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Not a duplicate; all content not in stored copy

</td></tr><tr style="mso-yfti-irow: 9;"><td style="width: 159.6pt; border: solid black 1.0pt; border-top: none; mso-border-top-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65 &amp; index

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65<span style="mso-spacerun: yes;"> </span>1970

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Not a duplicate; all content not in stored copy

</td></tr><tr style="mso-yfti-irow: 10; mso-yfti-lastrow: yes;"><td style="width: 159.6pt; border: solid black 1.0pt; border-top: none; mso-border-top-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65 &amp; supplement

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">v.65<span style="mso-spacerun: yes;"> </span>1970

</td><td style="width: 159.6pt; border-top: none; border-left: none; border-bottom: solid black 1.0pt; border-right: solid black 1.0pt; mso-border-top-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-alt: solid black .5pt; padding: 0in 5.4pt 0in 5.4pt;" valign="top" width="213">Not a duplicate; all content not in stored copy

</td></tr></tbody></table>

**Incoming volume:<span style="mso-spacerun: yes;"> </span>**new items that need to be processed.

**Stored Volume**:<span style="mso-spacerun: yes;"> </span>items already in storage.

3\. Procedures

A. Periodicals will be run through the deduplication process on unpacking, before sorting by size (there will be no need to sort duplicate volumes by size as they will not be processed into the storage facility).

B. For each periodical, a list of the volumes already in storage will be pulled up using the<span style="mso-bidi-font-weight: bold;"> [Journal Volume Overlap](https://grima.wrlc.org/Analytics/home.html) reports. The list will be scanned to determine if a volume in storage matches the volume in hand. The report is generated by scanning the volume’s barcode number (normally found inside the back cover; in same cases it will be on the outside of the cover). </span>

<span style="mso-bidi-font-weight: bold;">1. Go to web page, [Journal Volume Overlap - Barcode](https://grima.wrlc.org/Analytics/ia_journal_lookup-barcode.php)</span>

<span style="mso-bidi-font-weight: bold;">2. Scan volume barcode</span>

<span style="mso-bidi-font-weight: bold;">3. Check the “Volume” column of the report to see if the same volume is listed (as per the guidelines in section 2 above). The report will be sorted by volume number.</span>

<span style="mso-bidi-font-weight: bold;">a. The sort is a computer sort rather than a numerical sort; all volumes beginning with 1 precede all volumes beginning with 2, etc. So the order will appear as:</span>

<span style="mso-bidi-font-weight: bold;"><span style="mso-tab-count: 1;"> </span><span style="mso-tab-count: 1;"> </span><span style="mso-tab-count: 1;"> </span>1, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 2, 20, 21, 22…..</span>

<span style="mso-bidi-font-weight: bold;"><span style="mso-tab-count: 3;"> </span>1, 10, 101, 2, 20, 201, 3, 30, 301…..</span>

<span style="mso-bidi-font-weight: bold;">4. If the same volume is listed on the report as the volume in hand, the volume is a duplicate and should be placed on the “Discards” cart to be disposed.</span>

<span style="mso-bidi-font-weight: bold;">Note: If the only volume listed on the report is from the same library as the volume in hand, then it is probably not a duplicate. Compare the barcode on the item to the barcode on the volume. If they match, the volume is not a duplicate. If they do not match, the volume is a duplicate.</span>

<span style="mso-bidi-font-weight: bold;">5. If the volume in hand is not listed on the report, then it is not a duplicate and should be placed on the “accessions” cart to be processed into storage.</span>

<span style="mso-bidi-font-weight: bold;">6. If there is any doubt (as in the fourth example in section 2) the volume should be placed on the “Review” cart so a supervisor can make the determination.</span>