Error: The 'charmap' codec is having trouble encoding a character which is causing issues

Before anyone criticizes me for asking the same question multiple times, I want to clarify that I have tried various suggestions from different threads, but none of them have effectively solved my issue.

import json
def parse(fn):
    results = []
    with open(fn) as f:
        json_obj = json.loads(open(fn).read())
        for r in json_obj["result"]:
            print(r["name"])

parse("wine.json")

I am simply attempting to access values in a JSON file through iteration. However, when encountering unicode characters, I consistently encounter this error message.

Traceback (most recent call last):
  File "json_test.py", line 9, in <module>
    parse("wine.json")
  File "json_test.py", line 7, in parse
    print(r["name"])
  File "C:\Python34\lib\encodings\cp850.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u201c' in position
15: character maps to <undefined>

Despite trying various encoding and decoding methods as suggested in other discussions, I continue to face the same issue. Any assistance would be greatly appreciated.

Answer №1

Everything works smoothly until you attempt to print the string. In order to successfully print a string, it needs to be converted from pure Unicode to the byte sequences that are supported by your output device. This conversion requires encoding to the appropriate character set, which Python has recognized as cp850 - the default for Windows Console.

From Python 3.4 onwards, you have the option to configure the Windows console to use UTF-8 by executing the following command in the command prompt:

chcp 65001

This solution should resolve your problem, provided that you have selected a font that includes the specific character needed.

Starting with Python 3.6, this adjustment is no longer required - Windows always had comprehensive support for Unicode in the console, and Python now leverages this instead of relying on the basic code page I/O. Unicode printing to the console now functions seamlessly.

Answer №2

To solve the issue temporarily, I resorted to using Unidecode as a possible solution (unless someone has a better suggestion). However, this resulted in losing all accents, so I am still open to alternative fixes.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What is the reason behind appengine-rest-server returning JSON strings instead of integers for properties?

I set up a basic REST server on GAE utilizing the appengine-rest-server. The structure of my database is as follows: SomeString = db.StringProperty() SomeInt = db.IntegerProperty(default=0) SomeFloat = db.FloatProperty(default=-1.0) SomeDateTime = db.Dat ...

Unexpectedly large dataset for the Test and Training Sets

Currently, I am in the process of developing a predictive model using linear regression on a dataset containing 157673 records. The data is stored in a CSV file and follows this format: Timestamp,Signal_1,Signal_2,Signal_3,Signal_4,Signal_5 2021-04-13 ...

Creating a variable for an operator: What's the best way?

Is it possible to store the value 1 in the operator '+' as a variable in Python? I'm encountering an error while attempting this. My current project involves: current_number = int(input("Enter the number that you wish to be displayed: ")) p ...

Color Excel rows dynamically depending on their values (Identify multiple values)

I have a spreadsheet formatted like this: A B C D yesterday 70 today 70 yesterday 71 today 71 My objective is to compare the data for today and yesterday with matching IDs (70 and 71 in this instance) separated by blank rows indicating different ...

Creating a standalone executable file using pyInstaller

Is there a way to use the one folder mode of pyInstaller without collecting all dependencies every time? Typically, I find that I can reuse previously packaged dependencies and only need the executable. By stopping the process after building the .exe file ...

What is the best way to combine neighboring NumPy rows with matching values in the final column to create a consolidated row?

How can we solve this particular issue? I have an array with 30 rows containing columns (seq, high, low, status). The objective is to merge similar rows only when they are adjacent to each other in the sequence. The resulting row will have the maximum high ...

Searching for JSON data based on a specific key value filter using the jq query language

I am seeking assistance with filtering and extracting keys from a JSON file that contains key-value pairs. Specifically, I need to filter based on the inner key value pair within each key. Any guidance on how to accomplish this would be greatly appreciated ...

Exploring the DynamoDB List Data Type

Currently, I am working on an angular 8 application where I have chosen to store JSON data in a list data type within DynamoDB. Inserting records and querying the table for data has been smooth sailing so far. However, I have run into some challenges when ...

Creating a JSON hierarchy from an adjacency list

I am currently working with adjacency data that includes ID's and Parent ID's. My goal is to convert this data into hierarchical data by creating nested JSON structures. While I have managed to make it work, I encountered an issue when dealing ...

Unable to proceed with installation due to a ProtocolError message stating, "Connection aborted" along with a PermissionError of code 13, indicating that the process was denied permission

While attempting to use pip to install numpy or another package, I encountered the following error: PS C:\Users\giuse> pip install numpy WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection b ...

"Exploring the power of Ajax: a guide to automatically refreshing the response every

I am struggling to understand why this snippet of code isn't working as expected. The div updates when using .html, but not with my custom script. In my setup, I have two files: index.php and test.php The index file looks like this: $(document).rea ...

Converting Ordinal Time to Date-Time using Python

I need help converting a specific ordinal time array (shown below) to date-time format (YYYY-MM-DD hh:mm:ss.sss) in Anaconda/Python 3. I have searched for solutions to this issue, but none take into account the decimal precision of the original data. The ...

Guide to capturing the name H MATTHEWS from a website's HTML using Selenium with Python

When utilizing the 'contains' function, how can I extract specific information from this particular HTML structure? My goal is to gather the name "H MATTHEWS". HTML: <p> <strong>Date Published:</strong> &nbsp; 20 APRIL 2020 ...

Tips for parsing through extensive JSON documents containing diverse data types

In the process of developing an npm package that reads json files and validates their content against predefined json-schemas, I encountered issues when handling larger file sizes (50MB+). When attempting to parse these large files, I faced memory allocati ...

When I attempted to extract data with selenium through web scraping, the data stored in my csv file appeared unusual. Instead of the expected content, all I found were j

from selenium import webdriver from selenium.webdriver.chrome.service import Service from webdriver_manager.chrome import ChromeDriverManager from selenium.webdriver.common.keys import Keys from selenium.webdriver.common.by import By import time import cs ...

Is it possible to utilize the Compute Engine API with NodeJS to make changes to the files on a Compute Engine virtual machine?

Currently, I am working on a project in Google Cloud that involves utilizing both their App Engine and Compute Engine services. Within the Compute Engine, there is a virtual machine instance named "instance-1", where a python file (file.py) resides: name ...

"Looking to find the mean and standard deviation for each channel in the cifar10 dataset using PyTorch? Let's see

I am attempting to calculate the mean and standard deviation of each channel in the CIFAR10 dataset. Here is the code snippet I am using: import torch import torchvision import torchvision.transforms as transforms transform = transforms.Compose([transform ...

Create a JSON by merging two maps of strings

After examining the JSON String that I was expecting, here is what it looks like: { "startDate": "2013-01-01", "columns": "mode , event", "endDate": "2013-02-01", "selection": { "selectionMatch": "123456789012", "selectionT ...

Submitting data from a dropdown menu using Ajax in Struts 2: A step-by-step guide

I have a select tag with the s:select element inside a form. My goal is to send a post request to the specified action using Struts 2 json plugin. I do not have much knowledge in javascript or jquery. <s:form action="selectFileType" method="post" ...

Creating a dynamic MPTT structure with expand/collapse functionality in a Django template

I am looking for a way to display my MPTT model as a tree with dropdown capability (open/close nodes with children) and buttons that can expand/collapse all nodes in the tree with just one click. I have searched for examples, but the best I could find is ...