org.jsoup.Connection.get()方法的使用及代码示例

x33g5p2x  于2022-01-18 转载在 其他  
字(12.4k)|赞(0)|评价(0)|浏览(159)

本文整理了Java中org.jsoup.Connection.get()方法的一些代码示例，展示了Connection.get()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Connection.get()方法的具体详情如下：
包路径：org.jsoup.Connection
类名称：Connection
方法名：get

Connection.get介绍

[英]Execute the request as a GET, and parse the result.
[中]将请求作为GET执行，并解析结果。

代码示例

代码示例来源：origin: ChinaSilence/any-video

private String getOpenId(String accessToken) throws IOException{
  String url = openIdUri + accessToken;
  Document document = Jsoup.connect(url).get();
  String resultText = document.text();
  Matcher matcher = Pattern.compile("\"openid\":\"(.*?)\"").matcher(resultText);
  if (matcher.find()){
    return matcher.group(1);
  }
  return null;
}

代码示例来源：origin: deeplearning4j/dl4j-examples

/**
 * Get a list of all URLs in a page for zip files
 */
public static List<String> getZipUrlsFromPage(String url) {
  List<String> out = new ArrayList<>();
  try {
    Document doc = Jsoup.connect(url).get();
    Elements links = doc.select("a[href]");
    for (Element e : links) {
      String s = e.attr("href");
      if (s.endsWith(".zip")) {
        if (s.startsWith("http")) {
          //Absolute link
          out.add(s);
        } else {
          //Relative link
          out.add(e.baseUri() + s);
        }
      }
    }
  } catch (IOException e) {
    throw new RuntimeException(e);
  }
  return out;
}

代码示例来源：origin: RipMeApp/ripme

private String vscoImageToURL(String url) throws IOException{
  Document page = Jsoup.connect(url).userAgent(USER_AGENT)
                   .get();
  //create Elements filled only with Elements with the "meta" tag.
  Elements metaTags = page.getElementsByTag("meta");
  String result = "";
  for(Element metaTag : metaTags){
    //find URL inside meta-tag with property of "og:image"
    if (metaTag.attr("property").equals("og:image")){
      String givenURL = metaTag.attr("content");
      givenURL = givenURL.replaceAll("\\?h=[0-9]+", "");//replace the "?h=xxx" tag at the end of the URL (where each x is a number)
      
      result = givenURL;
      LOGGER.debug("Found image URL: " + givenURL);
      break;//immediately stop after getting URL (there should only be 1 image to be downloaded)
    }
  }
  
  //Means website changed, things need to be fixed.
  if (result.isEmpty()){
    LOGGER.error("Could not find image URL at: " + url);
  }
  
  return result;
  
}

代码示例来源：origin: loklak/loklak_server

/**
 * Article API
 * @param URL
 * @param JSONObject genericScraperData
 * @return genericScraperData
 */
public JSONObject articleAPI (String url, JSONObject genericScraperData) throws MalformedURLException{
  URL qurl = new URL(url);
  String data = "";
  try {
    data = null;// ArticleExtractor.INSTANCE.getText(qurl);
    genericScraperData.put("query", qurl);
    genericScraperData.put("data", data);
    genericScraperData.put("NLP", "true");
  }
  catch (Exception e) {
    if ("".equals(data)) {
      try {
        Document htmlPage = Jsoup.connect(url).get();
        data = htmlPage.text();
        genericScraperData.put("query", qurl);
        genericScraperData.put("data", data);
        genericScraperData.put("NLP", "false");
      } catch (Exception ex) {}
    }
  }
  return genericScraperData;
}

代码示例来源：origin: wangdan/AisenWeiBo

public static VideoBean getVideoFromWeipai(VideoBean video) throws Exception {
  Document dom = Jsoup.connect(video.getLongUrl()).get();
  video.setIdStr(KeyGenerator.generateMD5(video.getShortUrl()));
  Elements divs = dom.select("div[class=video_img WscaleH]");
  if (divs != null && divs.size() > 0) {
    video.setImage(divs.get(0).attr("data-url"));
  }
  divs = dom.select("video#video");
  if (divs != null && divs.size() > 0) {
    video.setVideoUrl(divs.get(0).attr("src"));
  }
  return video;
}

代码示例来源：origin: wangdan/AisenWeiBo

public static VideoBean getVideoFromSinaVideo(VideoBean video) throws Exception {
  Document dom = Jsoup.connect(video.getLongUrl()).get();
  video.setIdStr(KeyGenerator.generateMD5(video.getShortUrl()));
  Elements divs = dom.select("video.video");
  if (divs != null && divs.size() > 0) {
    String src = divs.get(0).attr("src");
    src = src.replace("amp;", "");
    video.setVideoUrl(src);
  }
  divs = dom.select("img.poster");
  if (divs != null && divs.size() > 0) {
    video.setImage(divs.get(0).attr("src"));
  }
  return video;
}

代码示例来源：origin: org.jsoup/jsoup

/**
 Fetch a URL, and parse it as HTML. Provided for compatibility; in most cases use {@link #connect(String)} instead.
 <p>
 The encoding character set is determined by the content-type header or http-equiv meta tag, or falls back to {@code UTF-8}.
 @param url           URL to fetch (with a GET). The protocol must be {@code http} or {@code https}.
 @param timeoutMillis Connection and read timeout, in milliseconds. If exceeded, IOException is thrown.
 @return The parsed HTML.
 @throws java.net.MalformedURLException if the request URL is not a HTTP or HTTPS URL, or is otherwise malformed
 @throws HttpStatusException if the response is not OK and HTTP response errors are not ignored
 @throws UnsupportedMimeTypeException if the response mime type is not supported and those errors are not ignored
 @throws java.net.SocketTimeoutException if the connection times out
 @throws IOException if a connection or read error occurs
 @see #connect(String)
 */
public static Document parse(URL url, int timeoutMillis) throws IOException {
  Connection con = HttpConnection.connect(url);
  con.timeout(timeoutMillis);
  return con.get();
}

代码示例来源：origin: wangdan/AisenWeiBo

public static VideoBean getVideoFromMeipai(VideoBean video) throws Exception {
  Document dom = Jsoup.connect(video.getLongUrl()).get();
  Elements divs = dom.select("div#mediaPlayer");
  if (divs != null && divs.size() > 0) {
    Element div = divs.get(0);
    video.setVideoUrl(div.attr("data-video"));
    video.setImage(div.attr("data-poster"));
  }
  video.setIdStr(KeyGenerator.generateMD5(video.getShortUrl()));
  return video;
}

代码示例来源：origin: ChinaSilence/any-video

private QQToken getToken(String tokenAccessApi) throws IOException{
  Document document = Jsoup.connect(tokenAccessApi).get();
  String tokenResult = document.text();
  String[] results = tokenResult.split("&");
  if (results.length == 3){
    QQToken qqToken = new QQToken();
    String accessToken = results[0].replace("access_token=", "");
    int expiresIn = Integer.valueOf(results[1].replace("expires_in=", ""));
    String refreshToken = results[2].replace("refresh_token=", "");
    qqToken.setAccessToken(accessToken);
    qqToken.setExpiresIn(expiresIn);
    qqToken.setRefresh_token(refreshToken);
    return qqToken;
  }
  return null;
}

代码示例来源：origin: RipMeApp/ripme

private JSONArray getPageUrls() {
  String postURL = "http://www.tsumino.com/Read/Load";
  try {
    // This sessionId will expire and need to be replaced
    cookies.put("ASP.NET_SessionId","c4rbzccf0dvy3e0cloolmlkq");
    Document doc = Jsoup.connect(postURL).data("q", getAlbumID()).userAgent(USER_AGENT).cookies(cookies).referrer("http://www.tsumino.com/Read/View/" + getAlbumID()).get();
    String jsonInfo = doc.html().replaceAll("<html>","").replaceAll("<head></head>", "").replaceAll("<body>", "").replaceAll("</body>", "")
        .replaceAll("</html>", "").replaceAll("\n", "");
    JSONObject json = new JSONObject(jsonInfo);
    return json.getJSONArray("reader_page_urls");
  } catch (IOException e) {
    LOGGER.info(e);
    sendUpdate(RipStatusMessage.STATUS.DOWNLOAD_ERRORED, "Unable to download album, please compete the captcha at http://www.tsumino.com/Read/Auth/"
        + getAlbumID() + " and try again");
    return null;
  }
}

代码示例来源：origin: jtablesaw/tablesaw

public String tableToCsv(String url) throws IOException {
    Document doc = Jsoup.connect(url).get();
    Elements tables = doc.select("table");
    if (tables.size() != 1) {
      throw new IllegalStateException(
          "Reading html to table currently works if there is exactly 1 html table on the page. "
              + " The URL you passed has " + tables.size()
              + ". You may file a feature request with the URL if you'd like your pagae to be supported");
    }
    Element table = tables.get(0);
    CsvWriterSettings settings = new CsvWriterSettings();
    StringWriter stringWriter = new StringWriter();
    CsvWriter csvWriter = new CsvWriter(stringWriter, settings);

    for (Element row : table.select("tr")) {
      Elements headerCells = row.getElementsByTag("th");
      Elements cells = row.getElementsByTag("td");
      String[] nextLine = Stream.concat(headerCells.stream(), cells.stream())
          .map(Element::text).toArray(String[]::new);
      csvWriter.writeRow(nextLine);
    }
    return stringWriter.toString();
  }
}

代码示例来源：origin: loklak/loklak_server

public static SusiThought locationWiseTime(String query) {
  
  Document html = null;
  JSONArray arr = new JSONArray();
  try {
    html = Jsoup.connect("http://www.timeanddate.com/worldclock/results.html?query=" + query).get();
  } catch (IOException e) {
    DAO.severe(e);
  }
  Elements locations = html.select("td");
  int i = 0;
  for (Element e : locations) {
    if (i % 2 == 0) {
      JSONObject obj = new JSONObject();
      String l = e.getElementsByTag("a").text();
      obj.put("location", l);
      String t = e.nextElementSibling().text();
      obj.put("time", t);
      arr.put(obj);
    }
    i++;
  }
  
  SusiThought json = new SusiThought();
  json.setData(arr);
  return json;
}

代码示例来源：origin: ChinaSilence/any-video

public static Document getDocWithPC(String url) {
  try {
    return Jsoup.connect(url).userAgent(UA_PC).timeout(TIME_OUT).ignoreContentType(true).get();
  } catch (IOException e) {
    log.error(ERROR_DESC + url);
    throw new AnyException(ERROR_DESC + url);
  }
}

代码示例来源：origin: RipMeApp/ripme

private static Document getDocument(String strUrl) throws IOException {
  return Jsoup.connect(strUrl)
              .userAgent(USER_AGENT)
              .timeout(10 * 1000)
              .maxBodySize(0)
              .get();
}

代码示例来源：origin: ChinaSilence/any-video

public static Document getDocWithPhone(String url) {
  try {
    return Jsoup.connect(url).userAgent(UA_PHONE).timeout(TIME_OUT).ignoreContentType(true).validateTLSCertificates(false).get();
  } catch (IOException e) {
    log.error(ERROR_DESC + url);
    throw new AnyException(ERROR_DESC + url);
  }
}

代码示例来源：origin: ChinaSilence/any-video

public static Document getDocWithPhone(String url, String cookie) {
  try {
    return Jsoup.connect(url).userAgent(UA_PHONE).timeout(TIME_OUT).header("Cookie", cookie).ignoreContentType(true).get();
  } catch (IOException e) {
    log.error(ERROR_DESC + url);
    throw new AnyException(ERROR_DESC + url);
  }
}

代码示例来源：origin: ChinaSilence/any-video

/**
   * 获取片段播放的 key
   */
  private String videoKey(String vid, String filename, String format) {
    try {
      Document document = Jsoup.connect(KEY_API).header("Cookie", COOKIE)
          .data("vid", vid).data("platform", PLATFORM)
          .data("otype", "json")
          .data("filename", filename).data("sdtfrom", SDTFROM)
          .data("format", format).data("guid", GUID).ignoreContentType(true).get();
      String result = document.text().replace("QZOutputJson=", "");
      System.out.println(result);
      result = result.substring(0, result.length() - 1);
      return JSONObject.parseObject(result).getString("key");
    } catch (IOException e) {
      log.info("request tencent video part api error, vid : " + vid);
      throw new AnyException("request tencent api error, vid : " + vid);
    }
  }
}

代码示例来源：origin: ChinaSilence/any-video

private Document requestAPI(String keyword) {
  try {
    return Jsoup.connect(api).userAgent(ua).ignoreContentType(true).data("wd", keyword).get();
  } catch (IOException e) {
    throw new AnyException(ExceptionEnum.VIDEO_SEARCH_ERROR);
  }
}

代码示例来源：origin: wangdan/AisenWeiBo

@Override
public String workInBackground(Void... p) throws TaskException {
  try {
    AccountBean accountBean = AccountUtils.getLogedinAccount();
    if (TextUtils.isEmpty(accountBean.getAccount()) || TextUtils.isEmpty(accountBean.getPassword()))
      throw new TaskException("", getString(R.string.account_fillaccount_faild));
    String js = FileUtils.readAssetsFile("mobile.js", GlobalContext.getInstance());
    js = js.replace("%username%", accountBean.getAccount());
    js = js.replace("%password%", accountBean.getPassword());
    Document dom = Jsoup.connect(url).get();
    String html = dom.toString();
    html = html.replace("</head>", js + "</head>");
    return html;
  } catch (Exception e) {
    e.printStackTrace();
  }
  throw new TaskException("", getString(R.string.account_fillaccount_faild));
}

代码示例来源：origin: ChinaSilence/any-video

/**
 * 调用腾讯接口，获取视频信息
 */
private String videoInfo(String vid) {
  try {
    Document document = Jsoup.connect(VIDEO_API).header("Cookie", COOKIE)
        .data("vids", vid).data("platform", PLATFORM)
        .data("sdtfrom", SDTFROM)
        .data("format", "10209")
        .data("otype", "json").data("defn", "fhd")
        .data("defaultfmt", "fhd").data("guid", GUID).ignoreContentType(true).get();
    String result = document.text().replace("QZOutputJson=", "");
    return result.substring(0, result.length() - 1);
  } catch (IOException e) {
    log.info("request tencent api error, vid : " + vid);
    throw new AnyException("request tencent api error, vid : " + vid);
  }
}

内容来源于网络，如有侵权，请联系作者删除！

Connection

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com