本文整理了Java中com.alibaba.datax.common.util.Configuration.get()
方法的一些代码示例,展示了Configuration.get()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Configuration.get()
方法的具体详情如下:
包路径:com.alibaba.datax.common.util.Configuration
类名称:Configuration
方法名:get
[英]根据用户提供的json path,寻址具体的对象。
NOTE: 目前仅支持Map以及List下标寻址, 例如:
对于如下JSON
{"a": {"b": {"c": [0,1,2,3]}}}
config.get("") 返回整个Map
config.get("a") 返回a下属整个Map
config.get("a.b.c") 返回c对应的数组List
config.get("a.b.c[0]") 返回数字0
[中]根据用户提供的json路径寻址具体的对象。
注:目前仅支持地图以及列表下标寻址, 例如:
对于如下JSON
{“a”:{“b”:{“c”:[0,1,2,3]}}
配置。得到(“”)返回整个地图
配置。得到(“a”)返回A.下属整个地图
配置。获取(“a.b.c”)返回C对应的数组列表
配置。获取(“a.b.c[0]”)返回数字0
代码示例来源:origin: ECNU-1X/DataX-Masking
/**
* 根据用户提供的json path,寻址Map对象,如果对象不存在,返回null
*/
@SuppressWarnings("unchecked")
public Map<String, Object> getMap(final String path) {
Map<String, Object> result = this.get(path, Map.class);
if (null == result) {
return null;
}
return result;
}
代码示例来源:origin: ECNU-1X/DataX-Masking
@Override
public void init() {
this.readerSliceConfig = super.getPluginJobConf();
this.userName = readerSliceConfig.getString(KeyConstant.MONGO_USER_NAME, readerSliceConfig.getString(KeyConstant.MONGO_USERNAME));
this.password = readerSliceConfig.getString(KeyConstant.MONGO_USER_PASSWORD, readerSliceConfig.getString(KeyConstant.MONGO_PASSWORD));
this.database = readerSliceConfig.getString(KeyConstant.MONGO_DB_NAME, readerSliceConfig.getString(KeyConstant.MONGO_DATABASE));
this.authDb = readerSliceConfig.getString(KeyConstant.MONGO_AUTHDB, this.database);
if(!Strings.isNullOrEmpty(userName) && !Strings.isNullOrEmpty(password)) {
mongoClient = MongoUtil.initCredentialMongoClient(readerSliceConfig,userName,password,authDb);
} else {
mongoClient = MongoUtil.initMongoClient(readerSliceConfig);
}
this.collection = readerSliceConfig.getString(KeyConstant.MONGO_COLLECTION_NAME);
this.query = readerSliceConfig.getString(KeyConstant.MONGO_QUERY);
this.mongodbColumnMeta = JSON.parseArray(readerSliceConfig.getString(KeyConstant.MONGO_COLUMN));
this.lowerBound = readerSliceConfig.get(KeyConstant.LOWER_BOUND);
this.upperBound = readerSliceConfig.get(KeyConstant.UPPER_BOUND);
this.isObjectId = readerSliceConfig.getBool(KeyConstant.IS_OBJECTID);
}
代码示例来源:origin: ECNU-1X/DataX-Masking
/**
* 合并其他Configuration,并修改两者冲突的KV配置
*
* @param another
* 合并加入的第三方Configuration
* @param updateWhenConflict
* 当合并双方出现KV冲突时候,选择更新当前KV,或者忽略该KV
* @return 返回合并后对象
*/
public Configuration merge(final Configuration another,
boolean updateWhenConflict) {
Set<String> keys = another.getKeys();
for (final String key : keys) {
// 如果使用更新策略,凡是another存在的key,均需要更新
if (updateWhenConflict) {
this.set(key, another.get(key));
continue;
}
// 使用忽略策略,只有another Configuration存在但是当前Configuration不存在的key,才需要更新
boolean isCurrentExists = this.get(key) != null;
if (isCurrentExists) {
continue;
}
this.set(key, another.get(key));
}
return this;
}
代码示例来源:origin: ECNU-1X/DataX-Masking
if (param.get(KEY_DATE) != null &&
(param.getLong(KEY_START_TIMESTAMP_MILLIS) != null || param.getLong(KEY_END_TIMESTAMP_MILLIS) != null) &&
(param.getLong(KEY_START_TIME_STRING) != null || param.getLong(KEY_END_TIME_STRING) != null)) {
if (param.get(KEY_DATE) != null &&
(param.getLong(KEY_START_TIMESTAMP_MILLIS) != null || param.getLong(KEY_END_TIMESTAMP_MILLIS) != null)) {
throw new OTSStreamReaderException("Can't set date and time range both, please check your config.");
if (param.get(KEY_DATE) != null &&
(param.getLong(KEY_START_TIME_STRING) != null || param.getLong(KEY_END_TIME_STRING) != null)) {
throw new OTSStreamReaderException("Can't set date and time range string both, please check your config.");
代码示例来源:origin: ECNU-1X/DataX-Masking
Configuration _conf = Configuration.from(con.toString());
if (Strings.isNullOrEmpty(_conf.getString("condition"))) {
query.put(_conf.getString("name"), _conf.get("value"));
} else {
query.put(_conf.getString("name"),
new BasicDBObject(_conf.getString("condition"), _conf.get("value")));
代码示例来源:origin: ECNU-1X/DataX-Masking
public static Configuration filterSensitiveConfiguration(Configuration configuration){
Set<String> keys = configuration.getKeys();
for (final String key : keys) {
boolean isSensitive = StringUtils.endsWithIgnoreCase(key, "password")
|| StringUtils.endsWithIgnoreCase(key, "accessKey");
if (isSensitive && configuration.get(key) instanceof String) {
configuration.set(key, configuration.getString(key).replaceAll(".", "*"));
}
}
return configuration;
}
代码示例来源:origin: ECNU-1X/DataX-Masking
@Override
public List<Configuration> split(int adviceNumber) {
LOG.debug("split() begin...");
List<Configuration> readerSplitConfigs = new ArrayList<Configuration>();
// 将每个单独的 object 作为一个 slice
List<String> objects = parseOriginObjects(readerOriginConfig
.getList(Constant.OBJECT, String.class));
if (0 == objects.size()) {
throw DataXException.asDataXException(
OssReaderErrorCode.EMPTY_BUCKET_EXCEPTION,
String.format(
"未能找到待读取的Object,请确认您的配置项bucket: %s object: %s",
this.readerOriginConfig.get(Key.BUCKET),
this.readerOriginConfig.get(Key.OBJECT)));
}
for (String object : objects) {
Configuration splitedConfig = this.readerOriginConfig.clone();
splitedConfig.set(Constant.OBJECT, object);
readerSplitConfigs.add(splitedConfig);
LOG.info(String.format("OSS object to be read:%s", object));
}
LOG.debug("split() ok and end...");
return readerSplitConfigs;
}
代码示例来源:origin: ECNU-1X/DataX-Masking
@Override
public void init() {
writerSliceConfig = super.getPluginJobConf();
this.writeMode = this.writerSliceConfig.getString(Key.WRITE_MODE);
this.schema = writerSliceConfig.getString(Key.SCHEMA);
this.table = writerSliceConfig.getString(Key.ADS_TABLE);
if(Constant.LOADMODE.equalsIgnoreCase(this.writeMode)) {
odpsWriterTaskProxy.setPluginJobConf(writerSliceConfig);
odpsWriterTaskProxy.init();
} else if(Constant.INSERTMODE.equalsIgnoreCase(this.writeMode) || Constant.STREAMMODE.equalsIgnoreCase(this.writeMode)) {
try {
this.tableInfo = AdsUtil.createAdsHelper(this.writerSliceConfig).getTableInfo(this.table);
} catch (AdsException e) {
throw DataXException.asDataXException(AdsWriterErrorCode.CREATE_ADS_HELPER_FAILED, e);
}
List<String> allColumns = new ArrayList<String>();
List<ColumnInfo> columnInfo = this.tableInfo.getColumns();
for (ColumnInfo eachColumn : columnInfo) {
allColumns.add(eachColumn.getName());
}
LOG.info("table:[{}] all columns:[\n{}\n].", this.writerSliceConfig.get(Key.ADS_TABLE), StringUtils.join(allColumns, ","));
AdsInsertUtil.dealColumnConf(writerSliceConfig, allColumns);
List<String> userColumns = writerSliceConfig.getList(Key.COLUMN, String.class);
this.columnNumber = userColumns.size();
} else {
throw DataXException.asDataXException(AdsWriterErrorCode.INVALID_CONFIG_VALUE, "writeMode 必须为 'load' 或者 'insert' 或者 'stream'");
}
}
代码示例来源:origin: ECNU-1X/DataX-Masking
if (lastPathKey.length() > 1 && lastPathKey.charAt(0) == '*'
&& lastPathKey.charAt(1) != '*') {
Object value = config.get(key);
if (value instanceof String) {
String newKey = key.substring(0, lastPathIndex)
代码示例来源:origin: ECNU-1X/DataX-Masking
/**
* 根据用户提供的json path,寻址List对象,如果对象不存在,返回null
*/
@SuppressWarnings("unchecked")
public List<Object> getList(final String path) {
List<Object> list = this.get(path, List.class);
if (null == list) {
return null;
}
return list;
}
代码示例来源:origin: ECNU-1X/DataX-Masking
/**
* 根据用户提供的json path,寻址String对象
*
* @return String对象,如果path不存在或者String不存在,返回null
*/
public String getString(final String path) {
Object string = this.get(path);
if (null == string) {
return null;
}
return String.valueOf(string);
}
代码示例来源:origin: ECNU-1X/DataX-Masking
/**
* 根据用户提供的json path,寻址List对象,如果对象不存在,返回null
*/
@SuppressWarnings("unchecked")
public <T> List<T> getList(final String path, Class<T> t) {
Object object = this.get(path, List.class);
if (null == object) {
return null;
}
List<T> result = new ArrayList<T>();
List<Object> origin = (List<Object>) object;
for (final Object each : origin) {
result.add((T) each);
}
return result;
}
代码示例来源:origin: ECNU-1X/DataX-Masking
/**
* 根据用户提供的json path,寻址Map对象,如果对象不存在,返回null;
*/
@SuppressWarnings("unchecked")
public <T> Map<String, T> getMap(final String path, Class<T> t) {
Map<String, Object> map = this.get(path, Map.class);
if (null == map) {
return null;
}
Map<String, T> result = new HashMap<String, T>();
for (final String key : map.keySet()) {
result.put(key, (T) map.get(key));
}
return result;
}
代码示例来源:origin: ECNU-1X/DataX-Masking
/**
* 根据用户提供的json path,寻址包含Configuration的Map,如果对象不存在,返回默认null
*/
@SuppressWarnings("unchecked")
public Map<String, Configuration> getMapConfiguration(final String path) {
Map<String, Object> map = this.get(path, Map.class);
if (null == map) {
return null;
}
Map<String, Configuration> result = new HashMap<String, Configuration>();
for (final String key : map.keySet()) {
result.put(key, Configuration.from(Configuration.toJSONString(map
.get(key))));
}
return result;
}
代码示例来源:origin: ECNU-1X/DataX-Masking
/**
* 根据用户提供的json path,寻址具体的对象,并转为用户提供的类型
* <p/>
* <br>
* <p/>
* NOTE: 目前仅支持Map以及List下标寻址, 例如:
* <p/>
* <br />
* <p/>
* 对于如下JSON
* <p/>
* {"a": {"b": {"c": [0,1,2,3]}}}
* <p/>
* config.get("") 返回整个Map <br>
* config.get("a") 返回a下属整个Map <br>
* config.get("a.b.c") 返回c对应的数组List <br>
* config.get("a.b.c[0]") 返回数字0
*
* @return Java表示的JSON对象,如果转型失败,将抛出异常
*/
@SuppressWarnings("unchecked")
public <T> T get(final String path, Class<T> clazz) {
this.checkPath(path);
return (T) this.get(path);
}
代码示例来源:origin: ECNU-1X/DataX-Masking
/**
* 删除path对应的值,如果path不存在,将抛出异常。
*/
public Object remove(final String path) {
final Object result = this.get(path);
if (null == result) {
throw DataXException.asDataXException(
CommonErrorCode.RUNTIME_ERROR,
String.format("配置文件对应Key[%s]并不存在,该情况是代码编程错误. 请联系DataX团队的同学.", path));
}
this.set(path, null);
return result;
}
代码示例来源:origin: ECNU-1X/DataX-Masking
/**
* 用户指定部分path,获取Configuration的子集
* <p/>
* <br>
* 如果path获取的路径或者对象不存在,返回null
*/
public Configuration getConfiguration(final String path) {
Object object = this.get(path);
if (null == object) {
return null;
}
return Configuration.from(Configuration.toJSONString(object));
}
代码示例来源:origin: ECNU-1X/DataX-Masking
public void init() {
try {
OTSStreamReaderConfig config = GsonParser.jsonToConfig(
(String) this.getPluginJobConf().get(OTSStreamReaderConstants.CONF));
StreamJob streamJob = StreamJob.fromJson(
(String) this.getPluginJobConf().get(OTSStreamReaderConstants.STREAM_JOB));
List<String> ownedShards = GsonParser.jsonToList(
(String) this.getPluginJobConf().get(OTSStreamReaderConstants.OWNED_SHARDS));
List<StreamShard> allShards = GsonParser.fromJson(
(String) this.getPluginJobConf().get(OTSStreamReaderConstants.ALL_SHARDS));
proxy.init(config, streamJob, allShards, new HashSet<String>(ownedShards));
} catch (TableStoreException ex) {
throw DataXException.asDataXException(new OTSReaderError(ex.getErrorCode(), "OTS ERROR"), ex.toString(), ex);
} catch (Exception ex) {
throw DataXException.asDataXException(OTSReaderError.ERROR, ex.toString(), ex);
}
}
代码示例来源:origin: ECNU-1X/DataX-Masking
/**
* 根据用户提供的json path,插入指定对象,并返回之前存在的对象(如果存在)
* <p/>
* <br>
* <p/>
* 目前仅支持.以及数组下标寻址, 例如:
* <p/>
* <br />
* <p/>
* config.set("a.b.c[3]", object);
* <p/>
* <br>
* 对于插入对象,Configuration不做任何限制,但是请务必保证该对象是简单对象(包括Map<String,
* Object>、List<Object>),不要使用自定义对象,否则后续对于JSON序列化等情况会出现未定义行为。
*
* @param path
* JSON path对象
* @param object
* 需要插入的对象
* @return Java表示的JSON对象
*/
public Object set(final String path, final Object object) {
checkPath(path);
Object result = this.get(path);
setObject(path, extractConfiguration(object));
return result;
}
内容来源于网络,如有侵权,请联系作者删除!