Reconciling two files in COBOL: matched, left-only, right-only
A complete GnuCOBOL match of two sorted files that reports every reference as matched, amount differs, bank only or book only, with control totals that must balance.
Two files that should agree and do not is the most common problem in batch processing. A bank statement against the company's own cash book, a supplier's invoice list against the purchase ledger, yesterday's master file against today's. The job is always the same: put each reference in one of four boxes (in both and equal, in both but different, only on the left, only on the right) and prove at the end that nothing was lost. This lesson does that with one COBOL program, two ten-record files and the balance-line pattern that mainframe shops have used for decades.
The listing is desk-checked against the GnuCOBOL 3.2 Programmer's Guide and was not compiled here, since cobc is not installed on the authoring machine. The expected report below was produced by running the Python companion in /code/cobol-reconcile-two-files.py, which reads the same two files, applies the same matching rules and prints the same edited layouts. The program builds on reading a sequential file to end of file and uses the edited pictures from printing money with edited pictures.
The two files
Both files use the same 47-byte layout and both are sorted by reference, lowest first. The sort order is not a detail: the whole method depends on it. One file is what the bank says happened, /datasets/cobol-reconcile-two-files-bank.dat; the other is what our own books say, /datasets/cobol-reconcile-two-files.dat.
| Field | Columns | PIC | Meaning |
|---|---|---|---|
| Ref | 1 to 8 | X(8) | Reference, eight digits, zero-padded |
| Date | 9 to 16 | 9(8) | YYYYMMDD |
| Amount | 17 to 27 | 9(9)V99 | Amount in cents, no decimal point |
| Desc | 28 to 47 | X(20) | Description |
Twelve references appear across the two files, ten in each, and they were chosen to hit every case. Six agree exactly. Two appear in both with different amounts: reference 10000104, where 876.50 on the bank side meets 867.50 in the books, the classic transposed-digit error, and 10000108, where a 200.00 taxi account went through the bank as 20.00. Two appear only at the bank, a service fee and an interest adjustment. Two appear only in the books, a cheque and a postage refill that the bank has not yet processed.
The core of the program
>>SOURCE FORMAT IS FREE
IDENTIFICATION DIVISION.
PROGRAM-ID. RECONCILE.
PROCEDURE DIVISION.
Main-Para.
OPEN INPUT Bank-File Book-File
OUTPUT Report-File
PERFORM Read-Bank
PERFORM Read-Book
PERFORM UNTIL WS-Bank-Key = HIGH-VALUES AND WS-Book-Key = HIGH-VALUES
EVALUATE TRUE
WHEN WS-Bank-Key = WS-Book-Key
PERFORM Same-Key
PERFORM Read-Bank
PERFORM Read-Book
WHEN WS-Bank-Key < WS-Book-Key
PERFORM Bank-Only
PERFORM Read-Bank
WHEN OTHER
PERFORM Book-Only
PERFORM Read-Book
END-EVALUATE
END-PERFORM
PERFORM Write-Totals
CLOSE Bank-File Book-File Report-File
STOP RUN.
That excerpt is the whole algorithm. The complete listing, with its file definitions, the report layouts, the read paragraphs and the totals, is in /code/cobol-reconcile-two-files.cob, about 230 lines. The read paragraph for the bank file shows the other half of the pattern:
Read-Bank.
READ Bank-File
AT END
MOVE HIGH-VALUES TO WS-Bank-Key
NOT AT END
IF Bank-Ref NOT > WS-Prev-Bank
DISPLAY "SEQUENCE ERROR IN BANK FILE AT " Bank-Ref
PERFORM Abort-Run
END-IF
MOVE Bank-Ref TO WS-Bank-Key WS-Prev-Bank
ADD Bank-Amount TO WS-Bank-Total
END-READ.
How the match works
The program holds one current record from each file and compares their keys. Three things can be true, and the EVALUATE has one branch for each.
If the keys are equal, both files hold the reference. Same-Key compares the amounts, writes a MATCHED or AMOUNT DIFFERS line, and the program reads the next record from both files.
If the bank key is lower, the bank has a reference the books do not, because the books have already moved past it or never had it. That is a BANK ONLY line, and only the bank file advances.
Otherwise the books have a reference the bank does not: BOOK ONLY, and only the book file advances.
The loop ends when both files are exhausted, and this is where HIGH-VALUES does its work. The guide defines HIGH-VALUE as "whatever character occupies the highest position in the program's collating sequence", and says HIGH-VALUES is a synonym. When a file reaches its end, its key is set to HIGH-VALUES, which compares higher than any real reference. So once the bank file is finished, every remaining book record compares lower than the bank key and falls through to BOOK ONLY, and the other way round. Nothing special has to be written for "one file finished early". The loop stops when both keys are HIGH-VALUES.
The keys are held in PIC X(8) fields rather than PIC 9(8) for the same reason: HIGH-VALUES is a character value, and keeping the key alphanumeric keeps every comparison a plain character comparison. Because every reference is exactly eight digits, the character order and the numeric order are the same.
Sequence checking
The method only works on sorted input. If a file arrived out of order, a reference could be reported as BANK ONLY and BOOK ONLY at the same time, and the report would look perfectly normal. So each read paragraph remembers the last key it accepted and stops the run if the next one is not higher. The previous-key fields start at LOW-VALUES so that the first record always passes.
A strict "higher than" also rejects duplicates, which is deliberate: this program treats a reference as unique within each file. A file where one reference can appear several times, such as partial payments against one invoice, needs a different design, usually a pre-pass that totals each reference.
When the check fails, Abort-Run moves 16 to RETURN-CODE and stops. The guide explains that main programs pass a code back to the operating system "simply by MOVEing a value to the RETURN-CODE special register", which is how a job scheduler would know this step failed rather than produced an empty report. A file that will not open ends the run with 12 the same way.
Reading to the end
The AT END clause, in the guide's words, runs its statement "if the statement fails due to a file status of 10 (end-of-file)". It also says an AT END clause "will not detect other non-zero file-status values", which is why each file has a FILE STATUS field and the open is checked explicitly. Both input files are LINE SEQUENTIAL: the guide says the runtime strips the line ending from each record and pads a short record with spaces, so a line saved without trailing blanks still reads as 47 bytes.
The report
The difference column and the three totals use a floating minus sign, -,---,---,--9.99. The guide notes that the minus sign "may also be used as floating editing symbols at the beginning of the picture-string", so the sign prints immediately to the left of the first digit rather than in a fixed column. The amount columns use BLANK WHEN ZERO, which is how a BANK ONLY line leaves the book column empty: the program moves zero into it and the field prints as blanks.
Worked example
Running the companion on the two files gives the report the program should write to recon.txt:
RECONCILIATION: BANK FILE AGAINST BOOK FILE
STATUS REFERENCE BANK AMOUNT BOOK AMOUNT DIFFERENCE
MATCHED 10000101 1,250.00 1,250.00
MATCHED 10000102 49.95 49.95
BOOK ONLY 10000103 300.00 -300.00
AMOUNT DIFFERS 10000104 876.50 867.50 9.00
MATCHED 10000105 12,000.00 12,000.00
BANK ONLY 10000106 15.00 15.00
MATCHED 10000107 450.00 450.00
AMOUNT DIFFERS 10000108 20.00 200.00 -180.00
MATCHED 10000109 999.99 999.99
BOOK ONLY 10000110 75.25 -75.25
BANK ONLY 10000111 3.50 3.50
MATCHED 10000112 2,500.00 2,500.00
MATCHED 6
AMOUNT DIFFERS 2
BANK ONLY 2
BOOK ONLY 2
BANK FILE TOTAL 18,164.94
BOOK FILE TOTAL 18,692.69
NET DIFFERENCE -527.75
The totals are the part to check first, because they prove the run is complete. Six plus two plus two plus two is twelve, the number of distinct references across both files. And the net difference must equal the sum of the difference column: 9.00 minus 180.00 plus 15.00 plus 3.50 minus 300.00 minus 75.25 is minus 527.75, which is exactly 18,164.94 minus 18,692.69. The companion asserts both of those before it finishes. If a record were skipped or counted twice, one of them would fail.
The standard output line the program displays at the end is MATCHED 00006 DIFFERS 00002 BANK ONLY 00002 BOOK ONLY 00002.
Compile and run
From the site folder, with GnuCOBOL 3.x installed:
cobc -x code/cobol-reconcile-two-files.cob
./cobol-reconcile-two-files
python code/cobol-reconcile-two-files.py
Compare recon.txt with the companion's output. To see the sequence check work, swap two lines in either data file and run again: the program should print a SEQUENCE ERROR line and exit with status 16.
Limitations
- Not compiled here. The listing was checked against the GnuCOBOL 3.2 Programmer's Guide, not by a compiler, and the expected report comes from the Python companion. If your cobc disagrees with anything above, the compiler wins.
- One record per reference. Duplicates are treated as a sequence error. Many real reconciliations need a many-to-one match, which this does not do.
- Exact amounts only. Any difference, even one cent, is reported. Real matching rules often allow a tolerance or match on date as well, and they belong in
Same-Key. - Trailing blanks. The companion prints each report line without trailing spaces; whether
recon.txtkeeps them depends on the runtime's line-sequential settings.
Index terms cobol, gnucobol, batch, file-matching, reconciliation