Invalid XML characters from ERP

Problem:
Data exchange between Sana and the ERP fails. This can occur on several places but is most found with indexing tasks (Scheduled tasks).
The error reported looks similalar to:
There is an error in XML document (2, 69249). 
'­', hexadecimal value 0x1F, is an invalid character. Line 2, position 69249..

The impact would be that any proces dependant on this data transfer cannot complete succesfully. 

Cause:
This error is thrown by the WCF client on the Sana (.NET) side. In NAV and AX it’s possible for string fields to contain non-readable Unicode characters, like the one mentioned in the error above: http://www.fileformat.info/info/unicode/char/001f/index.htm These <control> characters are not allowed in XML and WCF is throwing this exception while validating/parsing the response from the NAV\AX web service.

Applies to:
All Sana Commerce versions real-time connected to an ERP system.

Resolution:
These characters should not be present in NAV\AX. There can be different reasons for how they get there in the first place, but to solve this problem, the charchters should be removed.

To find out which field(s) contain (an) invalid character(s), follow these steps:
  1. Turn on XML request/response logging in NAV\AX.
  2. Reproduce the problem on the website
  3. Find the latest XML response files in the logging folder as configured in step 1
  4. Open the response files with for example Visual studio.
  5. Open the error list in Visual studio.
  6. Now see the records indicated in the error list.
  7. In the structure you can identify the ID for the record.
  8. Look up this record in the ERP system and update the broken value that appeard in the errorlist.

Feedback and Knowledge Base