# How to check duplicate data **URL:** \ **Category:** Forum\ **Tags:** Adabas-Natural, Natural, Natural-on-Mainframes\ **Created:** [June 5, 2008, 4:59pm UTC](https://techcommunity.softwareag.com/t/how-to-check-duplicate-data/76878 "2008-06-05T16:59:43Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![ROHAN\_LOBO](https://avatars.discourse-cdn.com/v4/letter/r/df705f/32.png) [@ROHAN\_LOBO](https://techcommunity.softwareag.com/u/ROHAN_LOBO)\ **Post date:** [June 5, 2008, 4:59pm UTC](https://techcommunity.softwareag.com/t/how-to-check-duplicate-data/76878/1 "2008-06-05T16:59:43Z")
Hi All, I want to know about checking duplicate data. My requirement is that :There are 2files(work files). Read file 1 and 2 Validate for error like Duplicate data in file, Non-Numeric check etc. If error encountered write to error file with the corresponding message. For example consider that there are 2 records with same data in file 1. How i can check for duplicates? Thanks, Rohan. ---
**Author:** ![Verne](https://sea1.discourse-cdn.com/techcommunity/user_avatar/techcommunity.softwareag.com/verne/32/89_2.png) [@Verne](https://techcommunity.softwareag.com/u/Verne)\ **Post date:** [June 6, 2008, 6:20pm UTC](https://techcommunity.softwareag.com/t/how-to-check-duplicate-data/76878/2 "2008-06-06T18:20:03Z")
Natural does not have a COMPARE function but it does not work like that… Try IF #FIELD1 = #FIELD2 WRITE (1) ‘Duplicate’ END-IF and maybe sort both work files first by the same key value to assist in the compare. ---
**Author:** ![Kazi1](https://avatars.discourse-cdn.com/v4/letter/k/b19c9b/32.png) [@Kazi1](https://techcommunity.softwareag.com/u/Kazi1)\ **Post date:** [June 7, 2008, 2:30pm UTC](https://techcommunity.softwareag.com/t/how-to-check-duplicate-data/76878/3 "2008-06-07T14:30:30Z")
I have written a program which checks for duplicates and runs once every year. The trick is sort both files. Read File 1 (preferably small file) move key to a variable, read file 2. If saved key is GT file 2 key read file 1 and if less read other way round and of course if equal write ‘error — record key’. Have fun. ---
**Author:** ![ROHAN\_LOBO](https://avatars.discourse-cdn.com/v4/letter/r/df705f/32.png) [@ROHAN\_LOBO](https://techcommunity.softwareag.com/u/ROHAN_LOBO)\ **Post date:** [June 9, 2008, 11:30am UTC](https://techcommunity.softwareag.com/t/how-to-check-duplicate-data/76878/4 "2008-06-09T11:30:41Z")
Thanks Kazi…I got your point. But in my case i want to eliminate the duplicates from the same file. For example: If file 1 is having same records, i have to eliminate. Requirement : Read file 1 and 2 Validate for error like Duplicate data in file, Non-Numeric check etc. If error encountered write to error file with the corresponding message. If every data looks okay then process the records.Need an output report sorted based on employee ID where employees are in Active status. Thanks, Rohan. ---
**Author:** ![Steve\_Robinson](https://sea1.discourse-cdn.com/techcommunity/user_avatar/techcommunity.softwareag.com/steve_robinson/32/107_2.png) [@Steve\_Robinson](https://techcommunity.softwareag.com/u/Steve_Robinson)\ **Post date:** [June 9, 2008, 4:28pm UTC](https://techcommunity.softwareag.com/t/how-to-check-duplicate-data/76878/5 "2008-06-09T16:28:34Z")
You are either describing your problem inadequately, or have never programmed. The two posted answers, by Verne and Kazi, both suffice to answer your problem as stated. Assuming you have a work file sorted by Employee ID, and you should not have duplicate Employee IDs, you can simply do an IF test between successive records. In Natural, you could also use AT BREAK to discover duplicates, or PERFORM BREAK, or even IF BREAK. steve ---
**Author:** ![Ralph\_Zbrog](https://sea1.discourse-cdn.com/techcommunity/user_avatar/techcommunity.softwareag.com/ralph_zbrog/32/78_2.png) [@Ralph\_Zbrog](https://techcommunity.softwareag.com/u/Ralph_Zbrog)\ **Post date:** [June 9, 2008, 8:08pm UTC](https://techcommunity.softwareag.com/t/how-to-check-duplicate-data/76878/6 "2008-06-09T20:08:50Z")
To delete (or simply identify) duplicate key values on a WORK file, use a sort utility (such as DFSORT or SYNCSORT) rather than Natural. SORT outperforms Natural when it comes to sequential I/O. To delete the duplicate records, your sort parameters will look similar to this: ```auto SORT FIELDS=(start,length,format,A),NOEQUALS SUM FIELDS=NONE ``` ---
**Author:** ![system](https://global.discourse-cdn.com/techcommunity/original/3X/c/d/cd19967bb605d28a72d77eb83a1cae22f3df2eb8.svg) [@system](https://techcommunity.softwareag.com/u/system)\ **Post date:** [April 2, 2021, 2:49pm UTC](https://techcommunity.softwareag.com/t/how-to-check-duplicate-data/76878/7 "2021-04-02T14:49:32Z")