# EDI Delimiter as part of the data **URL:** \ **Category:** Forum\ **Tags:** EDI\ **Created:** [July 13, 2006, 4:19am UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941 "2006-07-13T04:19:40Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![gedda\_mallik](https://avatars.discourse-cdn.com/v4/letter/g/f475e1/32.png) [@gedda\_mallik](https://techcommunity.softwareag.com/u/gedda_mallik)\ **Post date:** [July 13, 2006, 4:19am UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/1 "2006-07-13T04:19:40Z")
Hi, I came across an issue where sometimes the actual data contains the delimiter(in this case \*) as part of the data. Is there anyway to find the existence of this delimiter in any of the data elements before calling the convertToString call on the document. ``` Your reponse is greatly appreciated. ``` ---
**Author:** ![gupta\_r.17495](https://avatars.discourse-cdn.com/v4/letter/g/9fc348/32.png) [@gupta\_r.17495](https://techcommunity.softwareag.com/u/gupta_r.17495)\ **Post date:** [July 13, 2006, 7:35pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/2 "2006-07-13T19:35:57Z")
You can retreive the delimiters info from EDITPA from TN.Define a partner specific TPA and getEDITPA will have delimiters and map it to your convertToString. HTH, RMG ---
**Author:** ![reamon](https://avatars.discourse-cdn.com/v4/letter/r/b77776/32.png) [@reamon](https://techcommunity.softwareag.com/u/reamon)\ **Post date:** [July 13, 2006, 7:46pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/3 "2006-07-13T19:46:41Z")
One approach is to identify the fields that might contain delimiter characters and then do a replace on each of them to either change each character to a non-delimiter character–such as \* to \_ or space–or delete the character from the data. As rmg points out, you can retrieve the delimiters for a partner first and then base your find/replace on those delimiters instead of hard-coding. ---
**Author:** ![Jim\_Plested](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@Jim\_Plested](https://techcommunity.softwareag.com/u/Jim_Plested)\ **Post date:** [July 13, 2006, 10:36pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/4 "2006-07-13T22:36:44Z")
So that I understand the question. Are you creating outbound X12 from a flat file or some other type of data that may contain symbols such as |, \*, :? OR Is some knucklehead sending delimiters embeded within the elements? in the former, I imagine if you have a way of determining what outbound TP it is from the flat file, you could look up the delimiters in the agreement like RMG mentions. That way you dont send out bogus X12. However, in the latter, if an “_" were included within an REF03 as part of a description, where that partner is using "_” as the element separator. I think you need to reject the document and ask the TP to do a compliance check before sending. This may just be my FED/DOD background talking, but you should never try to code for poorly formed X12. Since there is no limit to how badly a partner can attempt to create it, and it can lead to a mis interpretation. I dont see how you could distinguish a delimiter from a wildcard in X12 received. ---
**Author:** ![gupta\_r.17495](https://avatars.discourse-cdn.com/v4/letter/g/9fc348/32.png) [@gupta\_r.17495](https://techcommunity.softwareag.com/u/gupta_r.17495)\ **Post date:** [July 14, 2006, 12:03am UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/5 "2006-07-14T00:03:13Z")
I agree with your comments on “\*” delimiter if this present in the segments/fields say MSG or REF03 etc…as we also know it is common used delimiter in the EDI tsets used as a FieldSeperator…This can lead X12 parsing issues etc… HTH, RMG ---
**Author:** ![Guest](https://sea1.discourse-cdn.com/techcommunity/user_avatar/techcommunity.softwareag.com/guest/32/75_2.png) [@Guest](https://techcommunity.softwareag.com/u/Guest)\ **Post date:** [July 14, 2006, 12:37am UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/6 "2006-07-14T00:37:30Z")
I agree with Jim that X12 data that contains delimiters within the elements should be rejected, especially since X12 does not have allowances for a release character. I have faced the second scenario many times, as our back-end system uses part numbers that begin with a ‘\*’ character. I usually do as Rob suggests and pass the offending fields through the replace service in the mapping. In certain circumstances it may be possible to scrub the entire string at once before mapping it to an internal format IS document, if it arrives in a format that makes this possible. Tim ---
**Author:** ![gedda\_mallik](https://avatars.discourse-cdn.com/v4/letter/g/f475e1/32.png) [@gedda\_mallik](https://techcommunity.softwareag.com/u/gedda_mallik)\ **Post date:** [July 14, 2006, 10:15am UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/7 "2006-07-14T10:15:58Z")
Hi All, Once again thanks for your suggestions and comments. The webMethods server gets an xml document generated from one of our backend systems, which we map to x12\_4010\_850 and then use convertToString to generate a edi document. What is happening is due to some user error some of the fields randomly gets an edi delimiter and this completely messes up the generated edi document. I was thinking of the option which Rob was suggesting, but doing a replace on a field by field basis is a tedious job. I was thinking if there is anyway I can built a java service to traverse through the 850 record and replace the delimiter if it exists in any field. Please let me know if I’m moving in the wrong direction Thanks, Mallik. ---
**Author:** ![gupta\_r.17495](https://avatars.discourse-cdn.com/v4/letter/g/9fc348/32.png) [@gupta\_r.17495](https://techcommunity.softwareag.com/u/gupta_r.17495)\ **Post date:** [July 14, 2006, 6:34pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/8 "2006-07-14T18:34:24Z")
“What is happening is due to some user error some of the fields randomly gets an edi delimiter and this completely messes up the generated edi document.” Can you elaborate at what stage this delimiters are messing up?after convertToString? or before you map to 4010 850 IDATA document? HTH, RMG ---
**Author:** ![Guest](https://sea1.discourse-cdn.com/techcommunity/user_avatar/techcommunity.softwareag.com/guest/32/75_2.png) [@Guest](https://techcommunity.softwareag.com/u/Guest)\ **Post date:** [July 14, 2006, 7:20pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/9 "2006-07-14T19:20:56Z")
One possibility is to convert the document to an XML string, pass the string to the replace service, replacing the delimiters with nothing, and convert the resulting string back to the original document type. Tim ---
**Author:** ![Mary\_Tillman](https://avatars.discourse-cdn.com/v4/letter/m/3e96dc/32.png) [@Mary\_Tillman](https://techcommunity.softwareag.com/u/Mary_Tillman)\ **Post date:** [July 14, 2006, 7:32pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/10 "2006-07-14T19:32:08Z")
In my experience, when data from a back-end system contains delimiters, it is restricted to certain fields. For example, data from SAP may contain ‘\*’ in the Name and Address fields because that is valid data from an SAP perspective. So it is a data issue and it is most likely possible to narrow the number of fields for the search/replace to a reasonable number and resolve in the the wM mapping service prior to convertToString. If the delimiter in the XML document is truly random, it’s seems that it would be a programming bug related to how the XML document is being generated and should be corrected there. Just my two-cents worth. ---
**Author:** ![reamon](https://avatars.discourse-cdn.com/v4/letter/r/b77776/32.png) [@reamon](https://techcommunity.softwareag.com/u/reamon)\ **Post date:** [July 14, 2006, 9:03pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/11 "2006-07-14T21:03:35Z")
> [@gedda\_mallik](#): > > Hi All, > I was thinking of the option which Rob was suggesting, but doing > a replace on a field by field basis is a tedious job. > [@tillmanm](#): > > In my experience, when data from a back-end system contains delimiters, it is restricted to certain fields. It can seem tedious to replace on a field by field basis, but as Mary points out, the number of fields that this needs to be done to usually number in the single digits. There just aren’t that many to worry about. The usual suspects are address fields and free-form comment/description fields. Be careful if you use a technique where you convert the entire doc to a string of some format and back. That approach can be full of special cases that are hard to account for. I think there might be a “scrub record” service of some sort floating around on this forum or on Advantage that traverses an entire IData structure to do work similar to this. The advantage of this approach is that it is relatively easy to replace all offending chars with a service call or two. The down-side is that it is relatively expensive in terms of memory and processing time–especially when the vast majority of the data will never have a char to be replaced and the times where a char actually occurs in the data is usually relatively rare. ---
**Author:** ![gedda\_mallik](https://avatars.discourse-cdn.com/v4/letter/g/f475e1/32.png) [@gedda\_mallik](https://techcommunity.softwareag.com/u/gedda_mallik)\ **Post date:** [July 17, 2006, 11:17pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/12 "2006-07-17T23:17:59Z")
Hi All, Once again thanks for your suggestions and comments I have actually tried rob’s approach and once we do it for a limited number of fields then the delimiter pops up in some other field. I’m thinking the tims approach (convert to xml replace the delimiter and convert it back to record )to be something which would make the process much easier and simple to do But I’m open to other suggestions too ---
**Author:** ![wMusers.Com1](https://sea1.discourse-cdn.com/techcommunity/user_avatar/techcommunity.softwareag.com/wmusers.com1/32/170_2.png) [@wMusers.Com1](https://techcommunity.softwareag.com/u/wMusers.Com1)\ **Post date:** [July 17, 2006, 11:41pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/13 "2006-07-17T23:41:01Z")
EDI guys, what is the recommended approach when a partner can’t or won’t send you valid EDI documents? Why try to accomodate and appease a partner that is unwilling or incapable of sending valid EDI documents? You’re not working on the source of the problem, IMHO. No amount of technology bandaids will ever be enough to ensure the quality of service level that the business leaders of your company and your partner’s expect. Mark ---
**Author:** ![Guest](https://sea1.discourse-cdn.com/techcommunity/user_avatar/techcommunity.softwareag.com/guest/32/75_2.png) [@Guest](https://techcommunity.softwareag.com/u/Guest)\ **Post date:** [July 17, 2006, 11:55pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/14 "2006-07-17T23:55:19Z")
I don’t think the situation is this cut-and-dried. An XML file is created that contains data which is problematic only when transformed into an EDI document. It’s not that a partner is submitting invalid EDI documents. This is as much a issue with X12 as it is with the data. IMO, a system creating an XML doc shouldn’t have to know the detailed EDI specifications of the ultimate receiver of that document so as to avoid the use of certain characters. Tim ---
**Author:** ![reamon](https://avatars.discourse-cdn.com/v4/letter/r/b77776/32.png) [@reamon](https://techcommunity.softwareag.com/u/reamon)\ **Post date:** [July 18, 2006, 1:12am UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/15 "2006-07-18T01:12:16Z")
> [@gedda\_mallik](#): > > Hi All, > …(convert to xml replace the delimiter and convert it back to record )to be something which would make the process much easier and simple to do I strongly advise against this approach. What will you do when one of your partners wants to use \> as a delimiter (which is not completely out of the question)? ---
**Author:** ![Guest](https://sea1.discourse-cdn.com/techcommunity/user_avatar/techcommunity.softwareag.com/guest/32/75_2.png) [@Guest](https://techcommunity.softwareag.com/u/Guest)\ **Post date:** [July 18, 2006, 1:17am UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/16 "2006-07-18T01:17:13Z")
Although I suggested the approach, I agree with Rob in general on this. IMO, this is only (possibly) feasible in a very limited application. It would definitely be hazardous to incorporate this into a general processing method for EDI. As others have suggested, it is definitely advisable to look into fixing the problem at the source, if at all possible. Tim ---
**Author:** ![reamon](https://avatars.discourse-cdn.com/v4/letter/r/b77776/32.png) [@reamon](https://techcommunity.softwareag.com/u/reamon)\ **Post date:** [July 18, 2006, 1:19am UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/17 "2006-07-18T01:19:56Z")
> [@connertp](#): > > …a system creating an XML doc shouldn’t have to know the detailed EDI specifications of the ultimate receiver of that document so as to avoid the use of certain characters. Agreed. There are only two workable approaches to this issue, IMO: - Replace the characters in question on a field by field basis in the transformation service. Requires knowledge of the fields at design time. - Scrub the document in its entirety by traversing the document and performing a replace on each field. This approach doesn’t care what the fields are until run-time but simply traverses all fields that exist. The chars to replace should be read from the TPAs and never hard-coded. The issue becomes more fun if you do both X12 and EDIFACT–one will need to account for different approaches those standards have. ---
**Author:** ![Guest](https://sea1.discourse-cdn.com/techcommunity/user_avatar/techcommunity.softwareag.com/guest/32/75_2.png) [@Guest](https://techcommunity.softwareag.com/u/Guest)\ **Post date:** [July 18, 2006, 1:50am UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/18 "2006-07-18T01:50:58Z")
If you’re interested in the second approach, there is a service in WmSamples called walkAnIData that gives an example of traversing the fields of a document at run-time. Tim ---
**Author:** ![gedda\_mallik](https://avatars.discourse-cdn.com/v4/letter/g/f475e1/32.png) [@gedda\_mallik](https://techcommunity.softwareag.com/u/gedda_mallik)\ **Post date:** [July 24, 2006, 11:41pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/19 "2006-07-24T23:41:06Z")
Tim, That is exactly what I was looking. Not sure how I have overlooked that one. Thanks, Mallik. ---
**Author:** ![system](https://global.discourse-cdn.com/techcommunity/original/3X/c/d/cd19967bb605d28a72d77eb83a1cae22f3df2eb8.svg) [@system](https://techcommunity.softwareag.com/u/system)\ **Post date:** [April 2, 2021, 3:05pm UTC](https://techcommunity.softwareag.com/t/edi-delimiter-as-part-of-the-data/108941/20 "2021-04-02T15:05:27Z")