org.jsoup.Jsoup.connect()方法的使用及代码示例

x33g5p2x  于2022-01-21 转载在 其他  
字(11.6k)|赞(0)|评价(0)|浏览(547)

本文整理了Java中org.jsoup.Jsoup.connect()方法的一些代码示例，展示了Jsoup.connect()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Jsoup.connect()方法的具体详情如下：
包路径：org.jsoup.Jsoup
类名称：Jsoup
方法名：connect

Jsoup.connect介绍

[英]Creates a new Connection to a URL. Use to fetch and parse a HTML page.

Use examples:

  • Document doc = Jsoup.connect("http://example.com").userAgent("Mozilla").data("name", "jsoup").get();
  • Document doc = Jsoup.connect("http://example.com").cookie("auth", "token").post();
    [中]创建到URL的新连接。用于获取和解析HTML页面。
    使用示例：
  • Document doc = Jsoup.connect("http://example.com").userAgent("Mozilla").data("name", "jsoup").get();
  • Document doc = Jsoup.connect("http://example.com").cookie("auth", "token").post();

代码示例

代码示例来源：origin: jphp-group/jphp

  1. @Signature
  2. public static Connection connect(String url) {
  3.   return Jsoup.connect(url);
  4. }

代码示例来源：origin: ChinaSilence/any-video

  1. private String getOpenId(String accessToken) throws IOException{
  2.   String url = openIdUri + accessToken;
  3.   Document document = Jsoup.connect(url).get();
  4.   String resultText = document.text();
  5.   Matcher matcher = Pattern.compile("\"openid\":\"(.*?)\"").matcher(resultText);
  6.   if (matcher.find()){
  7.     return matcher.group(1);
  8.   }
  9.   return null;
  10. }

代码示例来源：origin: RipMeApp/ripme

  1. private String vscoImageToURL(String url) throws IOException{
  2.   Document page = Jsoup.connect(url).userAgent(USER_AGENT)
  3.                    .get();
  4.   //create Elements filled only with Elements with the "meta" tag.
  5.   Elements metaTags = page.getElementsByTag("meta");
  6.   String result = "";
  7.   for(Element metaTag : metaTags){
  8.     //find URL inside meta-tag with property of "og:image"
  9.     if (metaTag.attr("property").equals("og:image")){
  10.       String givenURL = metaTag.attr("content");
  11.       givenURL = givenURL.replaceAll("\\?h=[0-9]+", "");//replace the "?h=xxx" tag at the end of the URL (where each x is a number)
  12.       
  13.       result = givenURL;
  14.       LOGGER.debug("Found image URL: " + givenURL);
  15.       break;//immediately stop after getting URL (there should only be 1 image to be downloaded)
  16.     }
  17.   }
  18.   
  19.   //Means website changed, things need to be fixed.
  20.   if (result.isEmpty()){
  21.     LOGGER.error("Could not find image URL at: " + url);
  22.   }
  23.   
  24.   return result;
  25.   
  26. }

代码示例来源：origin: wangdan/AisenWeiBo

  1. public static VideoBean getVideoFromWeipai(VideoBean video) throws Exception {
  2.   Document dom = Jsoup.connect(video.getLongUrl()).get();
  3.   video.setIdStr(KeyGenerator.generateMD5(video.getShortUrl()));
  4.   Elements divs = dom.select("div[class=video_img WscaleH]");
  5.   if (divs != null && divs.size() > 0) {
  6.     video.setImage(divs.get(0).attr("data-url"));
  7.   }
  8.   divs = dom.select("video#video");
  9.   if (divs != null && divs.size() > 0) {
  10.     video.setVideoUrl(divs.get(0).attr("src"));
  11.   }
  12.   return video;
  13. }

代码示例来源：origin: wangdan/AisenWeiBo

  1. public static VideoBean getVideoFromSinaVideo(VideoBean video) throws Exception {
  2.   Document dom = Jsoup.connect(video.getLongUrl()).get();
  3.   video.setIdStr(KeyGenerator.generateMD5(video.getShortUrl()));
  4.   Elements divs = dom.select("video.video");
  5.   if (divs != null && divs.size() > 0) {
  6.     String src = divs.get(0).attr("src");
  7.     src = src.replace("amp;", "");
  8.     video.setVideoUrl(src);
  9.   }
  10.   divs = dom.select("img.poster");
  11.   if (divs != null && divs.size() > 0) {
  12.     video.setImage(divs.get(0).attr("src"));
  13.   }
  14.   return video;
  15. }

代码示例来源：origin: wangdan/AisenWeiBo

  1. public static VideoBean getVideoFromMeipai(VideoBean video) throws Exception {
  2.   Document dom = Jsoup.connect(video.getLongUrl()).get();
  3.   Elements divs = dom.select("div#mediaPlayer");
  4.   if (divs != null && divs.size() > 0) {
  5.     Element div = divs.get(0);
  6.     video.setVideoUrl(div.attr("data-video"));
  7.     video.setImage(div.attr("data-poster"));
  8.   }
  9.   video.setIdStr(KeyGenerator.generateMD5(video.getShortUrl()));
  10.   return video;
  11. }

代码示例来源：origin: ChinaSilence/any-video

  1. private QQToken getToken(String tokenAccessApi) throws IOException{
  2.   Document document = Jsoup.connect(tokenAccessApi).get();
  3.   String tokenResult = document.text();
  4.   String[] results = tokenResult.split("&");
  5.   if (results.length == 3){
  6.     QQToken qqToken = new QQToken();
  7.     String accessToken = results[0].replace("access_token=", "");
  8.     int expiresIn = Integer.valueOf(results[1].replace("expires_in=", ""));
  9.     String refreshToken = results[2].replace("refresh_token=", "");
  10.     qqToken.setAccessToken(accessToken);
  11.     qqToken.setExpiresIn(expiresIn);
  12.     qqToken.setRefresh_token(refreshToken);
  13.     return qqToken;
  14.   }
  15.   return null;
  16. }

代码示例来源：origin: RipMeApp/ripme

  1. private JSONArray getPageUrls() {
  2.   String postURL = "http://www.tsumino.com/Read/Load";
  3.   try {
  4.     // This sessionId will expire and need to be replaced
  5.     cookies.put("ASP.NET_SessionId","c4rbzccf0dvy3e0cloolmlkq");
  6.     Document doc = Jsoup.connect(postURL).data("q", getAlbumID()).userAgent(USER_AGENT).cookies(cookies).referrer("http://www.tsumino.com/Read/View/" + getAlbumID()).get();
  7.     String jsonInfo = doc.html().replaceAll("<html>","").replaceAll("<head></head>", "").replaceAll("<body>", "").replaceAll("</body>", "")
  8.         .replaceAll("</html>", "").replaceAll("\n", "");
  9.     JSONObject json = new JSONObject(jsonInfo);
  10.     return json.getJSONArray("reader_page_urls");
  11.   } catch (IOException e) {
  12.     LOGGER.info(e);
  13.     sendUpdate(RipStatusMessage.STATUS.DOWNLOAD_ERRORED, "Unable to download album, please compete the captcha at http://www.tsumino.com/Read/Auth/"
  14.         + getAlbumID() + " and try again");
  15.     return null;
  16.   }
  17. }

代码示例来源：origin: ChinaSilence/any-video

  1. public static Document getDocWithPC(String url) {
  2.   try {
  3.     return Jsoup.connect(url).userAgent(UA_PC).timeout(TIME_OUT).ignoreContentType(true).get();
  4.   } catch (IOException e) {
  5.     log.error(ERROR_DESC + url);
  6.     throw new AnyException(ERROR_DESC + url);
  7.   }
  8. }

代码示例来源：origin: RipMeApp/ripme

  1. private String getImageLinkFromDLLink(String url) {
  2.   try {
  3.     Connection.Response response = Jsoup.connect(url)
  4.         .userAgent(USER_AGENT)
  5.         .timeout(10000)
  6.         .cookies(cookies)
  7.         .followRedirects(false)
  8.         .execute();
  9.     String imageURL = response.header("Location");
  10.     LOGGER.info(imageURL);
  11.     return imageURL;
  12.     } catch (IOException e) {
  13.       LOGGER.info("Got error message " + e.getMessage() + " trying to download " + url);
  14.       return null;
  15.     }
  16. }

代码示例来源：origin: HotBitmapGG/bilibili-android-client

  1. public static Observable<BaseDanmakuParser> downloadXML(final String uri) {
  2.     return Observable.create((Observable.OnSubscribe<BaseDanmakuParser>) subscriber -> {
  3.       if (TextUtils.isEmpty(uri)) {
  4.         subscriber.onNext(new BaseDanmakuParser() {
  5.           @Override
  6.           protected IDanmakus parse() {
  7.             return new Danmakus();
  8.           }
  9.         });
  10.       }
  11.       ILoader loader = null;
  12.       try {
  13.         HttpConnection.Response rsp = (HttpConnection.Response)
  14.             Jsoup.connect(uri).timeout(20000).execute();
  15.         InputStream stream = new ByteArrayInputStream(BiliDanmukuCompressionTools.
  16.             decompressXML(rsp.bodyAsBytes()));
  17.         loader = DanmakuLoaderFactory.
  18.             create(DanmakuLoaderFactory.TAG_BILI);
  19.         loader.load(stream);
  20.       } catch (IOException | DataFormatException | IllegalDataException e) {
  21.         e.printStackTrace();
  22.       }
  23.       BaseDanmakuParser parser = new BiliDanmukuParser();
  24.       assert loader != null;
  25.       IDataSource<?> dataSource = loader.getDataSource();
  26.       parser.load(dataSource);
  27.       subscriber.onNext(parser);
  28.     }).subscribeOn(Schedulers.io());
  29.   }
  30. }

代码示例来源：origin: RipMeApp/ripme

  1. private static Document getDocument(String strUrl) throws IOException {
  2.   return Jsoup.connect(strUrl)
  3.               .userAgent(USER_AGENT)
  4.               .timeout(10 * 1000)
  5.               .maxBodySize(0)
  6.               .get();
  7. }

代码示例来源：origin: ChinaSilence/any-video

  1. public static Document getDocWithPhone(String url) {
  2.   try {
  3.     return Jsoup.connect(url).userAgent(UA_PHONE).timeout(TIME_OUT).ignoreContentType(true).validateTLSCertificates(false).get();
  4.   } catch (IOException e) {
  5.     log.error(ERROR_DESC + url);
  6.     throw new AnyException(ERROR_DESC + url);
  7.   }
  8. }

代码示例来源：origin: ChinaSilence/any-video

  1. public static Document getDocWithPhone(String url, String cookie) {
  2.   try {
  3.     return Jsoup.connect(url).userAgent(UA_PHONE).timeout(TIME_OUT).header("Cookie", cookie).ignoreContentType(true).get();
  4.   } catch (IOException e) {
  5.     log.error(ERROR_DESC + url);
  6.     throw new AnyException(ERROR_DESC + url);
  7.   }
  8. }

代码示例来源：origin: ChinaSilence/any-video

  1. /**
  2.    * 获取片段播放的 key
  3.    */
  4.   private String videoKey(String vid, String filename, String format) {
  5.     try {
  6.       Document document = Jsoup.connect(KEY_API).header("Cookie", COOKIE)
  7.           .data("vid", vid).data("platform", PLATFORM)
  8.           .data("otype", "json")
  9.           .data("filename", filename).data("sdtfrom", SDTFROM)
  10.           .data("format", format).data("guid", GUID).ignoreContentType(true).get();
  11.       String result = document.text().replace("QZOutputJson=", "");
  12.       System.out.println(result);
  13.       result = result.substring(0, result.length() - 1);
  14.       return JSONObject.parseObject(result).getString("key");
  15.     } catch (IOException e) {
  16.       log.info("request tencent video part api error, vid : " + vid);
  17.       throw new AnyException("request tencent api error, vid : " + vid);
  18.     }
  19.   }
  20. }

代码示例来源：origin: ChinaSilence/any-video

  1. private Document requestAPI(String keyword) {
  2.   try {
  3.     return Jsoup.connect(api).userAgent(ua).ignoreContentType(true).data("wd", keyword).get();
  4.   } catch (IOException e) {
  5.     throw new AnyException(ExceptionEnum.VIDEO_SEARCH_ERROR);
  6.   }
  7. }

代码示例来源：origin: RipMeApp/ripme

  1. private void defaultSettings() {
  2.   this.retries = Utils.getConfigInteger("download.retries", 1);
  3.   connection = Jsoup.connect(this.url);
  4.   connection.userAgent(AbstractRipper.USER_AGENT);
  5.   connection.method(Method.GET);
  6.   connection.timeout(TIMEOUT);
  7.   connection.maxBodySize(0);
  8. }

代码示例来源：origin: org.jsoup/jsoup

  1. /**
  2.  * Prepare to submit this form. A Connection object is created with the request set up from the form values. You
  3.  * can then set up other options (like user-agent, timeout, cookies), then execute it.
  4.  * @return a connection prepared from the values of this form.
  5.  * @throws IllegalArgumentException if the form's absolute action URL cannot be determined. Make sure you pass the
  6.  * document's base URI when parsing.
  7.  */
  8. public Connection submit() {
  9.   String action = hasAttr("action") ? absUrl("action") : baseUri();
  10.   Validate.notEmpty(action, "Could not determine a form action URL for submit. Ensure you set a base URI when parsing.");
  11.   Connection.Method method = attr("method").toUpperCase().equals("POST") ?
  12.       Connection.Method.POST : Connection.Method.GET;
  13.   return Jsoup.connect(action)
  14.       .data(formData())
  15.       .method(method);
  16. }

代码示例来源：origin: wangdan/AisenWeiBo

  1. @Override
  2. public String workInBackground(Void... p) throws TaskException {
  3.   try {
  4.     AccountBean accountBean = AccountUtils.getLogedinAccount();
  5.     if (TextUtils.isEmpty(accountBean.getAccount()) || TextUtils.isEmpty(accountBean.getPassword()))
  6.       throw new TaskException("", getString(R.string.account_fillaccount_faild));
  7.     String js = FileUtils.readAssetsFile("mobile.js", GlobalContext.getInstance());
  8.     js = js.replace("%username%", accountBean.getAccount());
  9.     js = js.replace("%password%", accountBean.getPassword());
  10.     Document dom = Jsoup.connect(url).get();
  11.     String html = dom.toString();
  12.     html = html.replace("</head>", js + "</head>");
  13.     return html;
  14.   } catch (Exception e) {
  15.     e.printStackTrace();
  16.   }
  17.   throw new TaskException("", getString(R.string.account_fillaccount_faild));
  18. }

代码示例来源：origin: ChinaSilence/any-video

  1. /**
  2.  * 调用腾讯接口，获取视频信息
  3.  */
  4. private String videoInfo(String vid) {
  5.   try {
  6.     Document document = Jsoup.connect(VIDEO_API).header("Cookie", COOKIE)
  7.         .data("vids", vid).data("platform", PLATFORM)
  8.         .data("sdtfrom", SDTFROM)
  9.         .data("format", "10209")
  10.         .data("otype", "json").data("defn", "fhd")
  11.         .data("defaultfmt", "fhd").data("guid", GUID).ignoreContentType(true).get();
  12.     String result = document.text().replace("QZOutputJson=", "");
  13.     return result.substring(0, result.length() - 1);
  14.   } catch (IOException e) {
  15.     log.info("request tencent api error, vid : " + vid);
  16.     throw new AnyException("request tencent api error, vid : " + vid);
  17.   }
  18. }

内容来源于网络，如有侵权，请联系作者删除！

Jsoup

相关文章

    查看更多

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com