我是hadoop的初学者，我被告知要创建一个定制的inputformat类来读取json数据，我已经在Google上搜索并学习了如何创建一个定制的inputformat类来读取文件中的数据。但是我一直在解析json数据。我的json数据如下
[
{
"_count": 30,
"_start": 0,
"_total": 180,
"values": [
{
"attachment": {
"contentDomain": "techcarnival2013.eventbrite.com",
"contentUrl": "http://techcarnival2013.eventbrite.com/",
"imageUrl": "http://ebmedia.eventbrite.com/s3-s3/static/images/django/logos/eb_home_tm-trans-fb.png",
"summary": "Get to know a few thousand of Silicon Valley's best and brightest while enjoying unparalleled access to Candlestick Park,\u00a0games, food, music and more. We'll have carnival games you haven't played since you were ten, giant inflatable obstacle...",
"title": "Tech Carnival @ Candlestick Park"
},
"comments": {
"_total": 0
},
"creationTimestamp": 1373908436000,
"creator": {
"firstName": "Clayton",
"headline": "Director of Operations",
"secondname":{
"name":"myname"
},
"lastName": "K.",
"pictureUrl": "http://m.c.lnkd.licdn.com/mpr/mprx/0_R7Vm6_RqBDHaHCDzJHRA6hsNcwOfECjzMeaA6heqHeo0v6ovBWoCe8pVJiYrd5pJVu4KdbnQQ3Lj"
},
"likes": {
"_total": 0
},
"relationToViewer": {
"availableActions": {
"_total": 7,
"values": [
{
"code": "add-comment"
},
{
"code": "categorize-as-job"
},
{
"code": "categorize-as-promotion"
},
{
"code": "flag-as-inappropriate"
},
{
"code": "follow"
},
{
"code": "like"
},
{
"code": "reply-privately"
}
]
},
"isFollowing": false,
"isLiked": false
},
"summary": "Network with 4,000+ from the tech community, including folks from DFJ, Google, LinkedIn, Square, Uber, Y Combinator, 500 Startups, etc. $10 ticket gets you all-you-can-ride access to the pop-up Tech Carnival, will be the biggest Wednesday night of the tech summer.",
"title": "Tech Event @ Candlestick Park on Wednesday, July 17th! Come play carnival games with ~4,000 of the Bay area's best and brightest!"
},
{
"attachment": {
"contentDomain": "lifebeyondnumbers.com",
"contentUrl": "http://bit.ly/10VTqMu",
"imageUrl": "http://lifebeyondnumbers.com/wp-content/uploads/2013/07/lurnq_Online_Courses.jpg",
"summary": "LurnQ offers a platform for learning and teaching that is free for everyone. It caters to a diverse online audience and is relevant to everyone in general. The key segment that we address now is of life long learners.",
"title": "LurnQ - making lifelong learning clutter free, fun and a social..."
},
"comments": {
"_total": 0
},
"creationTimestamp": 1373883177000,
"creator": {
"firstName": "Syed",
"headline": "Founder and CEO at QubiqSquare",
"lastName": "Muksit",
"pictureUrl": "http://m.c.lnkd.licdn.com/mpr/mprx/0_Y5gdzlRCbQBTqIa-pXYnz-01b6KinDO-pFWnz-ZCZLk1WWdt-_SLUt2uWmrpzo0OxQxcVv6pRjbE"
},
"likes": {
"_total": 0
},
"relationToViewer": {
"availableActions": {
"_total": 7,
"values": [
{
"code": "add-comment"
},
{
"code": "categorize-as-job"
},
{
"code": "categorize-as-promotion"
},
{
"code": "flag-as-inappropriate"
},
{
"code": "follow"
},
{
"code": "like"
},
{
"code": "reply-privately"
}
]
},
"isFollowing": false,
"isLiked": false
},
"summary": "LurnQ offers a platform for learning and teaching that is free for everyone. It caters to a diverse online audience and is relevant to everyone in general. The key segment that we address now is of life long learners.",
"title": "There is so much to learn and most of the times, we don\u2019t even know that this-and-that good stuff exists. http://bit.ly/10VTqMu"
},
{
"attachment": {
"contentDomain": "techcarnival2013.eventbrite.com",
"contentUrl": "http://techcarnival2013.eventbrite.com/",
"imageUrl": "http://ebmedia.eventbrite.com/s3-s3/static/images/django/logos/eb_home_tm-trans-fb.png",
"summary": "Get to know a few thousand of Silicon Valley's best and brightest while enjoying unparalleled access to Candlestick Park,\u00a0games, food, music and more. We'll have carnival games you haven't played since you were ten, giant inflatable obstacle...",
"title": "Tech Carnival @ Candlestick Park"
},
"comments": {
"_total": 0
},
"creationTimestamp": 1373654758000,
"creator": {
"firstName": "Clayton",
"headline": "Director of Operations",
"lastName": "K.",
"pictureUrl": "http://m.c.lnkd.licdn.com/mpr/mprx/0_R7Vm6_RqBDHaHCDzJHRA6hsNcwOfECjzMeaA6heqHeo0v6ovBWoCe8pVJiYrd5pJVu4KdbnQQ3Lj"
},
"likes": {
"_total": 0
},
"relationToViewer": {
"availableActions": {
"_total": 7,
"values": [
{
"code": "add-comment"
},
{
"code": "categorize-as-job"
},
{
"code": "categorize-as-promotion"
},
{
"code": "flag-as-inappropriate"
},
{
"code": "follow"
},
{
"code": "like"
},
{
"code": "reply-privately"
}
]
},
"isFollowing": false,
"isLiked": false
},
"summary": "Network with 4,000+ from the tech community, including folks from DFJ, Google, LinkedIn, Square, Uber, Y Combinator, 500 Startups, etc. $10 ticket gets you all-you-can-ride access to the pop-up Tech Carnival, will be the biggest Wednesday night of the tech summer.",
"title": "Tech Event @ Candlestick Park on Wednesday, July 17th! Come play carnival games with ~4,000 of the Bay area's best and brightest!"
}
..........
........ so on
]所以我对如何读取自定义inputformat类中的json对象感到困惑。对如何解析它有什么想法吗？我想读取json数组中的单个json对象，我的意思是读取正确的json字符串，然后将字符串提供给map，我将在map中使用json解析器来构建自己的键值对。有什么帮助吗？提前谢谢
1条答案
按热度按时间lkaoscv71#
如果你的问题与马加姆拉维的评论一致，那么答案是好的。
但是，如果您有一个包含上面提到的所有json数据的文件，那么您可能希望读取整个文件，并从map函数中的值部分（byteswritable值）将其作为字符串检索，然后将其提供给同一map（）函数中可用的json解析器。
请看一下完整的文件输入格式
此外，如果在一个文件中有多个json对象数据，以及如何在Map器中获取每个json对象数据作为值，那么可以使用类似于xmlinputformat的东西，并定义开始和结束标记。对于json，必须有一个唯一的start和end标记，该标记准确地标记所需单个json数据对象的开始和结束。仅仅，如果您希望将上面的整个json对象作为值返回，那么使用start tag=“[{”和end tag=“}]”可能没有帮助，因为您已经有许多嵌套的对象会混淆inputformat。
如果在任何情况下都无法实现上述功能，请尝试构建自定义textinputformat，以覆盖在textinputformat中定义的linereader。
在linereader类中，您可以使用这两个集合（我可能有点过时，请检查现在是否可以使用配置属性配置，我知道cdh已经使其可配置，如果不需要重写的话）
您可以放开cr并将lf改为poing to“]\n[”，因为每个独立的json数据都将采用如图所示的格式，否则您将更清楚地知道如何使用它？
[
…json 1
]
[
…json 2
]
[
…json编号
]
（注意：在]和[之间有一个\n标记为不同json对象数据之间的边界）。
希望这有意义。