I've been struggling to send a UTF-8 encoded JSON string over TCP (Python 2.7). Below are some attempts and their results. The 'response' variable contains the JSON string I'm trying to send:
response = {"candidates":{"P":[{"mentionname":"Beyoncé","guess":[{"name":"BEYONCÉ","score":"1.00","eid":"72437"}]}],"E":[]}}
Attempt 1:
self.request.sendall(json.dumps(response+"\n", ensure_ascii=False))
This results in an error:
'ascii' codec can't encode character u'\xe9' in position 49: ordinal not in range(128)
Attempt 2:
self.request.sendall(json.dumps(response+"\n", encoding='utf8'))
This sends output to the other end (TCP client), but the last character of Beyoncé is incorrect:
{"candidates":{"P"[{"mentionname":"Beyoncé","guess":[{"name":"BEYONCÉ","score":"1.00","eid":"72437"}]}],"E":[]}}\n"
(The message is received on the client with message.decode('UTF-8')).
Attempt 3:
self.request.sendall(json.dumps(response+"\n", ensure_ascii=False, encoding='utf8'))
This results in an error:
'ascii' codec can't encode character u'\xe9' in position 49: ordinal not in range(128)
Attempt 4:
self.request.sendall(json.dumps(response+"\n").encode('utf8'))
This sends output to the other end (TCP client), but the last character of Beyoncé is incorrect:
{"candidates":{"P":[{"mentionname":"Beyoncé","guess":[{"name":"BEYONCÉ","score":"1.00","eid":"72437"}]}],"E":[]}}\n"
Attempt 5:
self.request.sendall(json.dumps(response+"\n", ensure_ascii=False).encode('utf8'))
This sends output to the other end, with the last character of Beyoncé as correct, but the double quotes are escaped:
{"candidates":{"P":[{"mentionname":"Beyoncé","guess":[{"name":"BEYONCÉ","score":"1.00","eid":"72437"}]}],"E":[]}}\n"
The last attempt is almost correct, except for the escaped double quotes. I understand this issue arises from "double encoding" but at the moment I have no choice but to use this workaround and remove backslashes in my TCP client code.
If anyone has a better solution, any help would be greatly appreciated! Regards, Patrick