Database Error Could Not Decode To Utf-8 Column
Contents |
here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss the workings and policies of this site About Us Learn more about Stack Overflow the company Business Learn more about hiring developers django could not decode to utf-8 column or posting ads with us Stack Overflow Questions Jobs Documentation Tags Users Badges Ask Question x sqlalchemy could not decode to utf-8 column Dismiss Join the Stack Overflow Community Stack Overflow is a community of 4.7 million programmers, just like you, helping each other. Join them; it only sqlite3 utf8 takes a minute: Sign up Python Encoding - Could not decode to utf8 up vote 0 down vote favorite I have an sqlite database that was populated by an external program. Im trying to read the data with python. When sqlite text_factory I attempt to read the data I get the following error: OperationalError: Could not decode to UTF-8 If I open the database in sqlite manager and look at the data in the offending record(s) using the inbuilt browse and search it looks fine, however if I export the table as csv, I notice the character £ in the offending records has become £ If I read the csv in python, the £ in the offending records is still read as
Sqlite Utf8
£ but its not a problem I can parse this manually. However I need to be able to read the data direct from the database, without the intermediate step of converting to csv. I have looked at some answers online for similar questions, I have so far tried setting "text_factory = str" and I have also tried changing the datatype of the column from TEXT to BLOB using sqlite manager, but still get the error. My code below results in the OperationalError: Could not decode to UTF-8 conn = sqlite3.connect('test.db') conn.text_factory = str curr = conn.cursor() curr.execute('''SELECT xml_dump FROM hands_1 LIMIT 5000 , 5001''') row = curr.fetchone() All the records above 5000 in the database have this character problem and hence produce the error. Any help appreciated. python sqlite csv utf-8 character-encoding share|improve this question asked May 7 '14 at 3:21 Java Fish 115 Did you try decoding it to UTF-16? –Natecat May 7 '14 at 3:34 I didn't, i'm new to programming, can you point me in the right direction of how I might try it? –Java Fish May 7 '14 at 3:38 Wait, why are you using a doc string there? –Natecat May 7 '14 at 3:43 Im sorry I don't understand the question? –Java Fish May 7 '14 at 3:50 If I try that I get an error : NameError: global name 'unicode' is not defined –Java
here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss the workings and policies of this site About Us Learn more about Stack Overflow
Python Sqlite3
the company Business Learn more about hiring developers or posting ads with us Stack python decode Overflow Questions Jobs Documentation Tags Users Badges Ask Question x Dismiss Join the Stack Overflow Community Stack Overflow is a community of 4.7 million programmers, just like you, helping each other. Join them; it only takes a minute: Sign up SQLite+SQLAlchemy+Python3: Could not decode to UTF-8 column up vote 1 down vote favorite I have a SQLAlchemy http://stackoverflow.com/questions/23508153/python-encoding-could-not-decode-to-utf8 model, in a Flask app, defined like this: class Food(Model): __tablename__ = 'food' id = db.Column(db.Integer, primary_key=True) short_desc = db.Column(db.String(80)) sci_name = db.Column(db.String(250)) I'm trying to perform a simple query using a pre-filled SQLite database, but when I call Food.query.all() I receive the error: (sqlite3.OperationalError) Could not decode to UTF-8 column 'food_sci_name' with text 'Actinidia chinensis �Zesy002� (Gold3)' I tried changing the engine text_factory to str, to bytes but none of these http://stackoverflow.com/questions/35751771/sqlitesqlalchemypython3-could-not-decode-to-utf-8-column changes have made any difference. python sqlite python-3.x flask-sqlalchemy share|improve this question asked Mar 2 at 15:41 Diego Ponciano 725817 add a comment| active oldest votes Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook. Your Answer draft saved draft discarded Sign up or log in Sign up using Google Sign up using Facebook Sign up using Email and Password Post as a guest Name Email Post as a guest Name Email discard By posting your answer, you agree to the privacy policy and terms of service. Browse other questions tagged python sqlite python-3.x flask-sqlalchemy or ask your own question. asked 7 months ago viewed 107 times Related 57SQLite, python, unicode, and non-utf data0Python Encoding - Could not decode to utf86Switching from SQLite to MySQL with Flask SQLAlchemy0Unable to insert values into SQLite using Flask and SQLAlchemy2Python (flask) BLOB image, SQLite, Sqlalchemy - display image0Sqlite using Python3 error?0Two Columns joining the same table with SqlAlchemy0Error adding columns to SQLITE2Flask-SQLAlchemy SQLITE boolean field always returns False0How to resolve utf-8 error when using odo function ? OperationalError: (sqlite3.OperationalError) Could not decode to UTF-8 column Hot Network Questions "Estas bele" vs "estas bela" (or "beli") Why do most log files use plain text rather than a binary
rowset = cur.fetchall() Traceback (most recent call last): File "
Support Search GitHub This repository Watch 64 Star 346 Fork 72 blaze/odo Code Issues 114 Pull requests 22 Projects 0 Pulse Graphs New issue OperationalError: (sqlite3.OperationalError) Could not decode to UTF-8 column #424 Open Rahuketu86 opened this Issue Feb 13, 2016 · 1 comment Projects None yet Labels None yet Milestone No milestone Assignees No one assigned 1 participant Rahuketu86 commented Feb 13, 2016 I am new to odo. I have a simple function for migrating databases: from blaze import Data, odo, like def migrate(source, target, table_names=None): src_db = Data(source) tbl_names = (src_db.fields, table_names)[table_names is not None] for tbl in tbl_names: display("Migrating Tablename ::" + tbl) odo? odo('{0}::{1}'.format(source, tbl), '{0}::{1}'.format(target, tbl)) migrate(url_config, url_config_bcp) Upon running this on a SQLite database. url_config = 'sqlite:///DB_CONFIG.db' url_config_bcp = 'sqlite:///DB_CONFIG_BCP.db' migrate(url_config, url_config_bcp) I get following error. How can I resolve this ? OperationalError: (sqlite3.OperationalError) Could not decode to UTF-8 column 'V_WIMS_WELLS$COMMENTS' with text ' 29/12/2013 : SAP issue related , WO# was closed but still apprear in WIMS for confirmation . WPH-41 Cut Surface CSG 9-5/8¿ on b P.S :- I am aware that we can set textfactory expilicitly in sqlalchemy db = create_engine('sqlite:///{0}'.format(sqlite_name)) conn = db.raw_connection() conn.connection.text_factory = str I don't know how to fix it within odo function Rahuketu86 commented Feb 13, 2016 ######################################################### A second attempt at writing function by passing metadata explicitly results in following error def migrate(source, target, table_names=None): src_db = Data(source) engine = sa.create_engine(source) conn = engine.raw_connection() conn.connection.text_factory = str metadata = sa.MetaData() metadata.bind = engine tbl_names = (src_db.fields,