试图通过JDBC将UTF-8插入到MySQL中时出现“字符串值不正确”?

ltskdhd1  于 2022-09-18  发布在  Java
关注(0)|答案(21)|浏览(229)

我的连接是这样设置的:
Connection conn = DriverManager.getConnection(url + dbName + "?useUnicode=true&characterEncoding=utf-8", userName, password);

在为表添加行时,我得到了以下错误:
Incorrect string value: 'xF0x90x8Dx83xF0x90...' for column 'content' at row 1

我插入了数千条记录,当文本包含\xF0(即不正确的字符串值始终以\xF0开头)时,我总是会收到这个错误。

该列的排序规则为utf8_General_ci。

这可能是什么问题?

a6b3iqyw

a6b3iqyw1#

I had this problem with my PLAY Java application. This is my stack trace for that exception:

javax.persistence.PersistenceException: Error[Incorrect string value: '\xE0\xA6\xAC\xE0\xA6\xBE...' for column 'product_name' at row 1]
  at io.ebean.config.dbplatform.SqlCodeTranslator.translate(SqlCodeTranslator.java:52)
  at io.ebean.config.dbplatform.DatabasePlatform.translate(DatabasePlatform.java:192)
  at io.ebeaninternal.server.persist.dml.DmlBeanPersister.execute(DmlBeanPersister.java:83)
  at io.ebeaninternal.server.persist.dml.DmlBeanPersister.insert(DmlBeanPersister.java:49)
  at io.ebeaninternal.server.core.PersistRequestBean.executeInsert(PersistRequestBean.java:1136)
  at io.ebeaninternal.server.core.PersistRequestBean.executeNow(PersistRequestBean.java:723)
  at io.ebeaninternal.server.core.PersistRequestBean.executeNoBatch(PersistRequestBean.java:778)
  at io.ebeaninternal.server.core.PersistRequestBean.executeOrQueue(PersistRequestBean.java:769)
  at io.ebeaninternal.server.persist.DefaultPersister.insert(DefaultPersister.java:456)
  at io.ebeaninternal.server.persist.DefaultPersister.insert(DefaultPersister.java:406)
  at io.ebeaninternal.server.persist.DefaultPersister.save(DefaultPersister.java:393)
  at io.ebeaninternal.server.core.DefaultServer.save(DefaultServer.java:1602)
  at io.ebeaninternal.server.core.DefaultServer.save(DefaultServer.java:1594)
  at io.ebean.Model.save(Model.java:190)
  at models.Product.create(Product.java:147)
  at controllers.PushData.xlsupload(PushData.java:67)
  at router.Routes$$anonfun$routes$1.$anonfun$applyOrElse$40(Routes.scala:690)
  at play.core.routing.HandlerInvokerFactory$$anon$3.resultCall(HandlerInvoker.scala:134)
  at play.core.routing.HandlerInvokerFactory$$anon$3.resultCall(HandlerInvoker.scala:133)
  at play.core.routing.HandlerInvokerFactory$JavaActionInvokerFactory$$anon$8$$anon$2$$anon$1.invocation(HandlerInvoker.scala:108)
  at play.core.j.JavaAction$$anon$1.call(JavaAction.scala:88)
  at play.http.DefaultActionCreator$1.call(DefaultActionCreator.java:31)
  at play.core.j.JavaAction.$anonfun$apply$8(JavaAction.scala:138)
  at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:655)
  at scala.util.Success.$anonfun$map$1(Try.scala:251)
  at scala.util.Success.map(Try.scala:209)
  at scala.concurrent.Future.$anonfun$map$1(Future.scala:289)
  at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
  at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
  at scala.concurrent.impl.CallbackRunnable.run$$$capture(Promise.scala:60)
  at scala.concurrent.impl.CallbackRunnable.run(Promise.scala)
  at play.core.j.HttpExecutionContext$$anon$2.run(HttpExecutionContext.scala:56)
  at play.api.libs.streams.Execution$trampoline$.execute(Execution.scala:70)
  at play.core.j.HttpExecutionContext.execute(HttpExecutionContext.scala:48)
  at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:68)
  at scala.concurrent.impl.Promise$KeptPromise$Kept.onComplete(Promise.scala:368)
  at scala.concurrent.impl.Promise$KeptPromise$Kept.onComplete$(Promise.scala:367)
  at scala.concurrent.impl.Promise$KeptPromise$Successful.onComplete(Promise.scala:375)
  at scala.concurrent.impl.Promise.transform(Promise.scala:29)
  at scala.concurrent.impl.Promise.transform$(Promise.scala:27)
  at scala.concurrent.impl.Promise$KeptPromise$Successful.transform(Promise.scala:375)
  at scala.concurrent.Future.map(Future.scala:289)
  at scala.concurrent.Future.map$(Future.scala:289)
  at scala.concurrent.impl.Promise$KeptPromise$Successful.map(Promise.scala:375)
  at scala.concurrent.Future$.apply(Future.scala:655)
  at play.core.j.JavaAction.apply(JavaAction.scala:138)
  at play.api.mvc.Action.$anonfun$apply$2(Action.scala:96)
  at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:304)
  at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)
  at scala.concurrent.impl.CallbackRunnable.run$$$capture(Promise.scala:60)
  at scala.concurrent.impl.CallbackRunnable.run(Promise.scala)
  at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
  at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
  at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
  at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
  at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
  at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:43)
  at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
  at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
  at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
  at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.sql.SQLException: Incorrect string value: '\xE0\xA6\xAC\xE0\xA6\xBE...' for column 'product_name' at row 1
  at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1074)
  at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4096)
  at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4028)
  at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2490)
  at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2651)
  at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2734)
  at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
  at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2458)
  at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2375)
  at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2359)
  at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
  at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
  at io.ebeaninternal.server.type.DataBind.executeUpdate(DataBind.java:82)
  at io.ebeaninternal.server.persist.dml.InsertHandler.execute(InsertHandler.java:122)
  at io.ebeaninternal.server.persist.dml.DmlBeanPersister.execute(DmlBeanPersister.java:73)
  ... 59 more

I was trying to save a record using io.Ebean. I fixed it by re creating my database with utf8mb4 collation, and applied play evolution to re create all tables so that all tables should be recreated with utf-8 collation.

CREATE DATABASE inventory CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
lfapxunr

lfapxunr2#

如果您在Java中遇到了类似的问题,并且不能灵活地更改数据库的字符集和排序编码,那么这个答案很适合您。
您可以使用Emoji Java库(如果您不使用Java,也可以使用类似的库)来实现相同的功能。您可以在保存/更新到数据库之前转换为别名,然后转换回Unicode POST保存/更新/从数据库加载。主要的好处是即使在编码之后文本的可读性也是如此,因为这个库只给表情符号起别名,而不是整个字符串。

代码快照示例:

字符串/Unicode到别名(保存/更新到数据库之前)

String str = "An 😀awesome 😃string with a few 😉emojis!";
String result = EmojiParser.parseToAliases(str);
System.out.println(result);
// Prints:
// "An :grinning:awesome :smiley:string with a few :wink:emojis!"

别名到Unicode/字符串(保存/更新/从数据库加载之后)

String str = "An :grinning:awesome :smiley:string 😄with a few :wink:emojis!";
String result = EmojiParser.parseToUnicode(str);
System.out.println(result);
// Prints:
// "An 😀awesome 😃string 😄with a few 😉emojis!"

注意:如果您使用的是Spring Boot,则可以在enitity中使用@PrePersist、@PreUpdate、@PostPersists、@PostUpdate、@PostLoad来执行别名和Unicode转换。

qlzsbp2j

qlzsbp2j3#

最后,在尝试了许多查询之后,这个方法奏效了

ALTER TABLE
table_name
CHANGE column_name column_name 
varchar(256)
CHARACTER SET utf8mb4
COLLATE utf8mb4_unicode_ci;
0yg35tkg

0yg35tkg4#

但是,重要的是要注意,MySQL连接器驱动程序版本必须早于5.1.47或更高版本。

efzxgjgh

efzxgjgh5#

我还必须删除并重新创建数据库的所有存储过程(以及函数),以便它们在utf8mb4的新字符集内执行。

运行:

SHOW PROCEDURE STATUS;

…查看哪些过程尚未更新为服务器的新CHARACTER_SET_CLIENT、COLLATION_CONNECTION和数据库归类的值。

qvk1mo1f

qvk1mo1f6#

删除模式并使用utf8mb4字符集重新创建模式解决了我的问题。

disho6za

disho6za7#

您需要在meta html中设置utf8mb4,还需要在服务器中更改tabel,并将排序规则设置为utf8mb4

72qzrwbm

72qzrwbm8#

如果您要创建一个新的MySQL表,您可以在创建时指定所有列的字符集,这为我解决了这个问题。

CREATE TABLE tablename (
<list-of-columns>
)
CHARSET SET utf8mb4 COLLATE utf8mb4_unicode_ci;

你可以阅读更多详细信息:https://dev.mysql.com/doc/refman/8.0/en/charset-column.html

dzhpxtsq

dzhpxtsq9#

my solution is change the column type from varchar(255) to blob

sxissh06

sxissh0610#

Hint: OnAWS RDSyou need a newParameter Groupfor your MySQL DB with the params (instead of editing a my.cnf)

  • collation_connection: utf8mb4_unicode_ci
  • collation_database: utf8mb4_unicode_ci
  • collation_server: utf8mb4_unicode_ci
  • character_set_client: utf8mb4
  • character_set_connection: utf8mb4
  • character_set_database: utf8mb4
  • character_set_results: utf8mb4
  • character_set_server: utf8mb4

Note: character_set_system stays "utf8"

These SQL commands doNOT**WORK PERMANENTLY - only in a session:

set character_set_server = utf8mb4;
set collation_server = utf8mb4_unicode_ci;
soat7uwm

soat7uwm11#

this is not the recommendation solution.. But worth to share. Since my project are upgrade the DBMS from old Mysql to newest (8). But I cant change the table structure, only the DBMS config (mysql). The solution for mysql server.

test onWindowsmysql8.0.15on mysqlconfigsearch for
sql-mode="....."

uncomment it. Or in my case just type/add

sql-mode="NO_ENGINE_SUBSTITUTION"

why not recommended solution. because if you use latin1 (my case).. the data insert successly but not the content (mysql not respond with error!!) . for example you type info like this

bla \x12

it save

bla [] (box)

okay.. for my problem.. I can change the field to UTF8.. But there is small problem.. see above answer about other solution is failed because the word is not inserted because contain more than 2 bytes (cmiiw).. this solution make your insert data become box. The reasonable is to use blob.. and you can skip my answer.

Another testing related to this were.. usingutf8_encodeon your code before save. I use on latin1 and it was success (I'm not usingsql-mode)! same as above answer usingbase64_encode.

My suggestion to analys your table requirement and tried to change from other format to UTF8

8ulbf1ek

8ulbf1ek12#

MySQL的utf8只允许使用UTF-8中可以用3个字节表示的Unicode字符。这里有一个需要4个字节的字符:\xF0\x90\x8D\x83(U+10343 GOTHIC LETTER SAUIL)。

如果您有MySQL5.5或更高版本,则可以将列编码从utf8更改为utf8mb4。此编码允许存储UTF-8中占用4个字节的字符。

您可能还必须在MySQL配置文件中将服务器属性character_set_server设置为utf8mb4。看来Connector/J defaults to 3-byte Unicode otherwise
例如,要在Connector/J中使用4字节UTF-8字符集,请使用character_set_server=utf8mb4配置MySQL服务器,并在Connector/J连接字符串中去掉characterEncoding。连接器/J随后将自动检测UTF-8设置。

06odsfpq

06odsfpq13#

I you only want to apply the change only for one field, you could try serializing the field

class MyModel < ActiveRecord::Base
  serialize :content

  attr_accessible :content, :title
end
ha5z0ras

ha5z0ras14#

Its mostly caused due to some unicode characters. In my case it was the Rupee currency symbol.

To quickly fix this, I had to spot the character causing this error. I copy pasted the entire text in a text editor like vi and replaced the troubling character with a text one.

ca1c2owp

ca1c2owp15#

Assuming you are usingphpmyadminto solve this error, follow these steps:

1.phpMyAdmin
1.your_table
1."Structure tab"

  1. change the Collation of your field from latin1_swedish_ci (or whatever it is) to utf8_general_ci

相关问题