WordPress REST API – when calling return VALID JSON but it fail to decode in PHP (AKA UTF-8 BOM \ufeff)

Quite interesting story, we have website running WordPress with two noticed problems:

  • After making call to REST API it return valid JSON response (debugged in Postman) but when trying raw json_decode in PHP it fail.
  • In PHP, after echoing (readfile) random file into client (wget), SHA1 sum of file on server and downloaded does NOT match, but content of downloaded file is exactly the same (tested via FileMerge)
  • Opening downloaded file in PHPStorm does not reveal any special characters in downloaded or original file

Trying this malformed CURL command reveal hidden from naked eye problem.

curl -d "auth=something" -X POST https://somewebsite.com/index.php\?rest_route\=/plugin/route/test --trace-ascii --

CURL pass raw output to terminal, terminal report unknown command (code was only used to diagnose) it’s malformed and sould not be used since it can execute arbitrary code!!!!

In front of JSON data there is UTF-8 BOM code (\ufeff) (it’s being ignored by most of application but not all, it should never be present in JSON response since encoding is send in HTTP header – some code is ignoring it, some will fail on decode JSON and thread it as invalid syntax.

When we know what to look for, we need scan all PHP files on server and locate ( 0xEF, 0xBB, 0xBF ) string in root website directory using following command:

grep -rlI $’\xEF\xBB\xBF’ .

From result ignore anything that is not .PHP since those files can have BOM and not directly parsed via PHP interpreter.

Download those PHP file and examine with “HexEditor” reveal BOM before PHP code that is being send along ANY content served from PHP.

Cleaning file from “BOM” mark (on MAC):

 sed $’1s/\xef\xbb\xbf//’ < in8sync-customization.php > clean-in8sync-customization.php

Compare two files together to make sure only BOM is removed.

Upload cleaned file in original location and repeat process if there is more than one. Be happy fixing unusual issue!!

Leave a Reply

Your email address will not be published.