Need serious help with HTTP in Java

[Mad 86]

Ill take a look, thanks!

Unfortunately, i'm not really looking for a standalone IDE or program. I want something like HTTPClient which uses their API

 

It comes as a Java driver, you add it as a library to your app and it allows you to get attributes of a page, modify it, etc 

[Cardozz 46]

So basically i run the program, do the sht i want to do and it generates the code for me? Or am i missing something ?

[slasso 27]

Well is it for a login? Is it a request you need to make and you already logged in?

 

In chrome, hit f12 to open up the network viewer. Submit the request through the website and view the request in that network tool. A POST request consists of 2 parts. The request header and request body.

 

You should see the URL you need to post to. "login.example.com/?abc=12345&pid=0" or something like that. The url you will post to is login.example.com.

URIBuilder uriBuilder = new URIBuilder("login.example.com");uriBuilder.addParameter("abc","12345"); // request params, anything after the /? and separated by &// same for pid 0

Then you set the request headers based on what you see in the request header section of the network tool.

RequestBuilder rb = RequestBuilder.create("POST");rb.addHeader("origin", "http://login.example.com");//add whatever ones like Accept, content type, etc. 

Then set the body similarly with information from your form you filled out based on the request body in the network tool output.

 

Then send the request. Check the status code, if it's what you expected do stuff.

 

On my phone so I can't say much more atm

[Mad 86]

So basically i run the program, do the sht i want to do and it generates the code for me? Or am i missing something ?

No, its just an external library you add to your project, that allows you to manipulate web pages.

 

For example, you can do like

driver.findElementByID('mad-is-a-beast).val();

would find that element and return its value, etc.

 

and you can maniuplate the web page by sending key strokes, setting attributes, etc

https://streamable.com/mmy9

[Cardozz 46]

Wow thanks both for the help! I'll try it out asap!

 

btw, @Mad, can you provide the link to the library? i can't seem to find it!

[Mad 86]

Wow thanks both for the help! I'll try it out asap!

 

btw, @Mad, can you provide the link to the library? i can't seem to find it!

http://goo.gl/IHP6Qw

http://selenium-release.storage.googleapis.com/2.53/selenium-java-2.53.0.zip

 

You need both these

[slasso 27]

Wow thanks both for the help! I'll try it out asap!

 

btw, @Mad, can you provide the link to the library? i can't seem to find it!

Np. I just did this for the Microsoft live login page. I can show you an example in a bit

 

 

Here's something I just did. What is currently does is, gets the login.live.com page with a GET. Then it uses information from that response to make a POST with user/password credentials to log in and receive a response back. So you'll get back two responses total.

package com.steve.checker.gt_checker;

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.CookieStore;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpUriRequest;
import org.apache.http.client.methods.RequestBuilder;
import org.apache.http.cookie.Cookie;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.nodes.Node;
import org.jsoup.select.Elements;

public class XboxHttpRequestSender {

	public static void main(String[] args) {
		try {
			CookieStore httpCookieStore = new BasicCookieStore(); // cookies here just in case we need it later
			HttpClientBuilder builder = HttpClientBuilder.create().setDefaultCookieStore(httpCookieStore);
			HttpClient client = builder.build();

			// get login page first
			RequestBuilder getRequestBuilder = RequestBuilder.create("GET").setUri("https://login.live.com");
			getRequestBuilder.addHeader("Connection", "keep-alive");
			getRequestBuilder.addHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");

			HttpUriRequest getLogin = getRequestBuilder.build();
			HttpResponse response = client.execute(getLogin);
			System.out.println(response.toString());

			String serverData = getServerData(response);

			int indexOfFirstCharPpft = serverData.indexOf("value=") + 7; // first PPFT char value
			int indexOfLastCharPpft = serverData.indexOf("\"/>'", indexOfFirstCharPpft);
			int indexOfFirstPostUrl = serverData.indexOf("urlPost:'") + 9;
			int indexOfLastPostUrl = serverData.indexOf('\'', indexOfFirstPostUrl);

			String postUrl = serverData.substring(indexOfFirstPostUrl, indexOfLastPostUrl);
			String ppft = serverData.substring(indexOfFirstCharPpft, indexOfLastCharPpft);


			// post now that we have everything we need.
			RequestBuilder requestPostBuilder = RequestBuilder.create("POST").setUri(postUrl);

			// headers first.
			requestPostBuilder.addHeader("Connection", "keep-alive");
			requestPostBuilder.addHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
			requestPostBuilder.addHeader("Content-Type", "application/x-www-form-urlencoded");

			String payload = setPayLoad(ppft);

			HttpEntity payloadEntity = new StringEntity(payload);
			requestPostBuilder.setEntity(payloadEntity);

			HttpUriRequest post = requestPostBuilder.build();

			HttpResponse postResponse = client.execute(post);
			System.out.println(postResponse.toString());

		} catch (ClientProtocolException e) {
			e.printStackTrace();
		} catch (IOException e) {
			e.printStackTrace();
		}
	}

	/**
	 * Get a json from embedded javascript inside live's login page. live's login POST requires 'PPFT' 
	 * variable which are inside one of the <script type='text/javascript'> tags in the html. Conveniently, the entire
	 * POST url is also in this string. With the request parameters set, so we can obtain it from here.
	 * 
	 * @[member='param'] response Response from the GET
	 * 
	 * @return String containing two variables we need
	 * 
	 * @[member='throwsies'] org.apache.http.ParseException
	 * @[member='throwsies'] IOException
	 */
	private static String getServerData(HttpResponse response) throws org.apache.http.ParseException, IOException {
		Document doc = Jsoup.parse(EntityUtils.toString(response.getEntity()));
		Elements scripts = doc.select("script"); // all the scripts tags.

		String serverData = null;

		for (Element e : scripts) {
			for (Node n : e.childNodes()) {
				if (n.toString().contains("PPFT")) { // the one that contains 'PPFT' has the entire JSON with 'uaid' too.
					serverData = n.toString();
					break;
				}
			}
		}

		return serverData;
	}

	/**
	 * Set the request body for the POST request to login.
	 * 
	 * @[member='param'] ppft some required random token variable embedded in the html page.
	 * 
	 * @return request body with all variables needed for the request, i13=12343&i22=32432&...
	 */
	private static String setPayLoad(String ppft) {
		// most of these variables I hardcode based on what was sent when I monitored the POST request in a browser.
		
		StringBuilder payload = new StringBuilder();
		try {
			String i16 = URLEncoder.encode(makeI16JsonStr(), "UTF-8");
			String i2 = String.valueOf(1);
			String i17 = String.valueOf(0);
			String i19 = String.valueOf(6829);
			String i21 = String.valueOf(0);
			String i22 = String.valueOf(0);
			String i13 = String.valueOf(0);
			String i18 = "__DefaultLogin_Strings%7C%2C__DefaultLogin_Core%7C%2C";

			String fspost = String.valueOf(0);
			String NewUser = String.valueOf(1);
			String LoginOptions = String.valueOf(3);
			String type = String.valueOf(11);

			String PPSX = "Passpo";

			String login = "examplename123%40hotmail.com";
			String loginfmt = "examplename123%40hotmail.com";
			String passwd = "mypassword123";

			payload.append("i16=" + i16 + "&").append("i2=" + i2 + "&").append("i17=" + i17 + "&").append("i19=" + i19 + "&").append("i22=" + i22 + "&")
			.append("i21=" + i21 + "&").append("i13=" + i13 + "&").append("i18=" + i18 + "&").append("fspost=" + fspost + "&")
			.append("NewUser=" + NewUser + "&").append("LoginOptions=" + LoginOptions + "&").append("FoundMSAs=\"\"&")
			.append("type=" + type + "&").append("PPSX=" + PPSX + "&").append("login=" + login + "&")
			.append("loginfmt=" + loginfmt + "&").append("passwd=" + passwd + "&").append("PPFT=" + URLEncoder.encode(ppft, "UTF-8"));

		} catch (UnsupportedEncodingException e) {
			e.printStackTrace();
		}

		return payload.toString();
	}

	/**
	 * This JSON String is a request body parameter that is required to make the POST. In an actual browser, each key/value 
	 * is obtained from a javascript variable embedded in the browser, ex: window.performance.timing.navigationstart. 
	 * Here I just create new timestamps for each one that I saw wasn't '0'. These can all be 0 too, as long as each key/value pair 
	 * is present.
	 * 
	 * @return String in JSON syntax with the 'i16' variable that is required.
	 */
	private static String makeI16JsonStr() {		
		return "{\"navigationStart\":" + System.currentTimeMillis() + 
				",\"unloadEventStart\":0,\"unloadEventEnd\":0,\"redirectStart\":0,\"redirectEnd\":0,\"fetchStart\":" + 
				System.currentTimeMillis() + ",\"domainLookupStart\":" + System.currentTimeMillis() + ",\"domainLookupEnd\":" + 
				System.currentTimeMillis() + ",\"connectStart\":" + System.currentTimeMillis() + ",\"connectEnd\":" + System.currentTimeMillis() + 
				",\"secureConnectionStart\":" + System.currentTimeMillis() + ",\"requestStart\":" + System.currentTimeMillis() + 
				",\"responseStart\":" + System.currentTimeMillis() + ",\"responseEnd\":" + System.currentTimeMillis() + ",\"domLoading\":" + 
				System.currentTimeMillis() + ",\"domInteractive\":" + System.currentTimeMillis() + ",\"domContentLoadedEventStart\":" + 
				System.currentTimeMillis() + ",\"domContentLoadedEventEnd\":" + System.currentTimeMillis() + ",\"domComplete\":" + 
				System.currentTimeMillis() + ",\"loadEventStart\":" + System.currentTimeMillis() + ",\"loadEventEnd\":0}";
	}

}

Edit these line with a valid email/password for live if you want to test it.

String login = "examplename123%40hotmail.com";
String loginfmt = "examplename123%40hotmail.com";
String passwd = "mypassword123";

It will return a 200 for the GET, and a 302 for the POST.This will allow access to xbox.com's main homepage and gametag checker/claim page, with the client authenticated as the user provided, and allowing to send a POST to change a gamertag.

 

 

 

I use maven to get the dependencies such as Apache HttpClient and jsoup. Here's the dependencies if you know how to build with maven.

<dependency>
	<groupId>org.apache.httpcomponents</groupId>
	<artifactId>httpclient</artifactId>
	<version>4.5.2</version>
</dependency>
<!-- http://mvnrepository.com/artifact/org.jsoup/jsoup -->
<dependency>
	<groupId>org.jsoup</groupId>
	<artifactId>jsoup</artifactId>
	<version>1.9.1</version>
</dependency>

[Cardozz 46]

Wow, thanks @slasso! This will definitely help me out. Very much appreciated man!

[Hopewelljnj 46]

This is making me want to mess with this stuff

[slasso 27]

Why is this in scripting support anyways?

 

Wow, thanks @slasso! This will definitely help me out. Very much appreciated man!

Np.

 

This is making me want to mess with this stuff

You should lol. I'm trying to make a gamertag changer