grand exchange api [Free] humblePhlipper: Grand Exchange flipping with automatic item selection (300+ gp/hr, 0 xp/hr)

[apnasus 49]

My first script, flips Logs and Death runes ANY ITEMS YOU CHOSE at the Grand Exchange - both these items have high volumes with consistently profitable margins and are f2p with no trade restrictions for new accounts.

Terminology

Spoiler

Bid price AKA buy price AKA insta-sell price AKA low price is the price we buy at

Ask price AKA sell price AKA insta-buy price AKA high price is the price we sell at

Mechanics

Spoiler

Update bid/ask prices every ~0.7 ~1-4 1.0, 10-20 seconds using the OSRS Wiki (not DreamBot) API and user selected pricing options.

1. Always sell any inventory of items we have bought at ask.
2. If and only if no inventory exists and spread is profitable, buy items at bid.
3. Stop script when all GE limits have been hit, timeout is triggered, profit cutoff is triggered, or user stops manually.
4. Save trade history, current config, and selection charcteristics locally ("/Dreambot/Scripts/humblePhlipper/History/") for analysis.

Requirements

Spoiler

* Stand by GE with some money

Features

Spoiler

Paint overlay with profit calculations after every logged sale, automatic 4hr GE limit tracking which carries over between sessions,

GUI supporting custom config profiles with automatic item selection and custom bid prioritisation,

Trades and end of session trade history CSVs output to console log,

Custom config profiles, 4hr limits linked to accounts, and trading session histories saved locally,

Toggle bidding off to exit gracefully by cancelling buy offers and selling off remaining inventory,

On the hour summaries of all the ongoing sessions ('instances') on your PC/VM sent to your discord webhook (learn how to generate a discord webhook URL here https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks),

As well as individual end of session summaries,

Post session inspection of (all) trade histories with summary stats,

Post session summary stats and regression,

`bidding == false` taking too long to sell off inventory? You can manually end a session without resorting to the stop button (which messes with profit calculations):

1) Press pause, let the script finish its loop, and cancel all sell offers.

2) Press resume and let the scipt collect, this way it logs partially fulfilled sell offers.

3) Press pause, let the script finish its loop, and sell of all the inventory just collected.

4) Press resume and let the script collect, this way it logs your manual sales.

5) Make sure that if it isn't already, bidding is toggled off, the scipt will exit after logging all trades.

Config

Spoiler

Config can be set by the GUI or QuickStart. Custom config profiles are saved by the GUI to "/Dreambot/Scripts/humblePhlipper/Config/" as JSONs. Config for each session is attached to the trade history for that session in "/Dreambot/Scripts/humblePhlipper/History/" to facilitate analysis.

Default config,

Spoiler

{
  "timeout": 240.0,
  "sysExit": false,
  "discordWebhook": null,
  "profitCutOff": 2147483647,
  "debug": false,
  "maxBidValue": 1000000,
  "maxBidVol": 20000,
  "noSelfCompeting": false,
  "priorityProfit": 100,
  "priorityVol": 100,
  "priorityCapitalBinding": 0,
  "pricing": "bestOfLatestFiveMinute",
  "pricingOffset": -1,
  "selections": [],
  "auto": true,
  "minVol": 10000,
  "maxBidAskVolRatio": 3.0,
  "minMargin": -10,
  "maxBidPrice": 20000,
  "tradeRestricted": false,
  "members": false,
  "numToSelect": 512
}

 

Paramater types,

Spoiler

{
  "timeout": Float,
  "sysExit": Boolean,
  "discordWebhook": String,
  "profitCutOff": Integer,
  "debug": Boolean,
  "maxBidValue": Integer,
  "maxBidVol": Integer,
  "noSelfCompeting": Boolean,
  "priorityProfit": Integer,
  "priorityVol": Integer,
  "priorityCapitalBinding": Integer,
  "pricing": String,
  "pricingOffset": Integer,
  "selections": Set<Integer>,
  "auto": Boolean,
  "minVol": Integer,
  "maxBidAskVolRatio": Float,
  "minMargin": Integer,
  "maxBidPrice": Integer,
  "tradeRestricted": Boolean,
  "members": Boolean,
  "numToSelect": Integer
}

 

Paramater supports if restricted,

Spoiler

{
  "priorityProfit": {0, 1, ..., 99, 100}
  "priorityVol": {0, 1, ..., 99, 100}
  "priorityCapitalBinding": {0, 1, ..., 99, 100}
  "pricing": {"latest", "fiveMinute", "oneHour", "bestOfLatestFiveMinute", "worstOfLatestFiveMinute"}
}

 

Parameter descriptions,

Spoiler

{
  "timeout": time in minutes after which to stop making buy offers, sell off remaining inventory, and exit,
  "sysExit": whether to close client on exit,
  "discordWebhook": discord webhook url, like https://discord.com/api/webhooks/XXX/XXX, to send notifications to,
  "profitCutOff": profit after which to stop making buy offers, sell off remaining inventory, and exit,
  "debug": whether to log debugging output to console log,
  "maxBidValue": maximum value (quantity * price) of any buy offer,
  "maxBidVol": maximum quantity of any buy offer,
  "noSelfCompeting": whether to avoid bidding on items other bot instances (on the same machine) are already trading,
  "priorityProfit": relative weight assigned to ordinal rank of post-tax profit margin in ordering selections,
  "priorityVol": relative weight assigned to ordinal rank of latest 1 hour volume in ordering selections,
  "priorityCapitalBinding": relative weight assigned to ordinal rank of bid * target, where target is 4hr_GE_limit - used_limit,  in ordering selections,
  "pricing": see below,
  "pricingOffset": see below,
  "selections": set of item IDs to be used if `auto = false`,
  "auto": select items automatically, overrides `selections` if true,
  "minVol": minimum 1 hour volume of candidate  selections,
  "maxBidAskVolRatio": maximum 1_hour_bid_volume / 1_hour_ask_volume of candidate automatic selections,
  "mingMargin": minimum post-tax profit margin of candidate automatic selections,
  "maxBidPrice": maximum bid of candaidate automatic selections,
  "tradeRestricted": whether the account has 20 hour / 10 quest point / 100 skill level trading restrictions,
  "members": whether the account is memebrs,
  "numToSelect": number of automatic selections (ordered according to priorityProfit / priorityVol / priorityCapitalBinding),
}

 

`pricing` description,,

Spoiler

{
  "latest": Latest bid/ask prices from OSRS Wiki API,
  "fiveMinute": Latest 5 minute average bid/ask prices from OSRS Wiki API,
  "oneHour": Latest 1 hour average bid/ask prices from OSRS Wiki API,
  "bestOfLatestFiveMinute": bid = min{latest_bid, fiveMinute_bid}, ask = max{latest_ask, fiveMinute_ask}
  "worstOfLatestFiveMinute": bid = max{latest_bid, fiveMinute_bid}, ask = min{latest_ask, fiveMinute_ask}
}

`bestOf` targets bigger profit margins, `worstOf` targets greater volume - I recommend the former here.

`pricingOffset` description,

Spoiler

bid = baseline_bid - pricingOffset
ask = baseline_ask + pricingOffset

Positive offsets target bigger profit margins, negative offsets target greater volume - I recommend the latter here.

Bandwith Usage

Spoiler

This script routinely makes HTTP requests to fetch price data. Prediced excess bandwith usage assuming 300 KB upper bound for price requests,

{"pricing" = "latest"} : 60.74 - 111.90 MB / hour
{"pricing" = "fiveMinute"} : 3.90 MB / hour
{"pricing" = "oneHour"} : 0.30 MB / hour
{"pricing" = "bestOfLatestFiveMinute"} : 60.74 - 111.90 MB / hour
{"pricing" = "worstOfLatestFiveMinute"} : 60.74 - 111.90 MB / hour

Instances running on the same PC/VM will attempt to connect to each other and share price data, with only 1 instance making HTTP requests. (So 10 instances should use no more excess bandwidth than 1.)

QuickStart

Spoiler

QuickStart accepts a single paramater, this can be a Config JSON or the name of a Config file in "/Dreambot/Scripts/humblePhlipper/Config/". Using no params will start with default config. Omitting the the -params option entirely will launch the GUI as usual.

Python example using a Config file name,

Spoiler

import subprocess
import re
import pandas as pd
from io import StringIO

SubprocessCommand = ["java", "-jar", "C:/Users/me/DreamBot/BotData/client.jar",
                     "-script", "humblePhlipper",
                     "-account", "my_account",
                     "-covert",
                     "-world", "308",
                     "-params", "<Custom 1>"]

# Open the subprocess with stdout as a pipe
process = subprocess.Popen(SubprocessCommand, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, shell=False)
stdout, stderr = process.communicate()

# After the trading session ends with `sysExit = true` read data
regex = "<trades>(.*?)</trades>"
df = pd.read_csv(StringIO(re.search(regex, stdout, re.DOTALL).group(1)))
df['size'] = df['vol'] * df['price']
print("Total profit = ", sum(df['size']))
df.groupby('name')['size'].sum().plot(kind='bar') # Bar chart of profit by item

 

Python example using a Config JSON,

Spoiler

import subprocess
import re
import pandas as pd
from io import StringIO

SubprocessCommand = ["java", "-jar", "C:/Users/me/DreamBot/BotData/client.jar",
                     "-script", "humblePhlipper",
                     "-account", "my_account",
                     "-covert",
                     "-world", "302",
                     "-params", """{
  "timeout": 240.0,
  "sysExit": true,
  "profitCutOff": 2147483647,
  "maxBidValue": 1000000,
  "maxBidVol": 10000,
  "priorityProfit": 100,
  "priorityVol": 0,
  "priorityCapitalBinding": 0,
  "pricing": "latest",
  "pricingOffset": 0,
  "apiInterval": 1,
  "selections": [1511, 569], // Logs and Death rune
  "auto": true,
  "minVol": 100000,
  "maxBidAskVolRatio": 3.0,
  "minMargin": -10,
  "maxBidPrice": 20000,
  "tradeRestricted": false,
  "members": false,
  "numToSelect": 64
}"""]


# Open the subprocess with stdout as a pipe
process = subprocess.Popen(SubprocessCommand, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, shell=False)
stdout, stderr = process.communicate()

# After the trading session ends with `sysExit = true` read data
regex = "<trades>(.*?)</trades>"
df = pd.read_csv(StringIO(re.search(regex, stdout, re.DOTALL).group(1)))
df['size'] = df['vol'] * df['price']
print("Total profit = ", sum(df['size'])
df.groupby('name')['size'].sum().plot(kind='bar') # Bar chart of profit by item

Regression Analysis

Spoiler

Base Model,

Spoiler

Y = XB + U

Y is an n x 1 matrix of profit per hour for `n` sessions.

X is an n x (k+1) matrix of k explanatory variables and an intercept (constant).

B is the matrix of coefficients to be estimated.

U is the matrix of errors.

Estimates,

Spoiler

OLS coefficients,

β = (X'WX)^{-1}X'WY = (X'X)^{-1}X'Y
W = diag(1, ..., 1)

WLS coefficients using runtimes as weights,

β = (X'WX)^{-1}X'WY
W = diag(runtime_1, ..., runtime_n)

Variance-covariance matrix of coefficients assuming homoskedasticity,

Var(β) = (1/n) * (E'WE) * (X'WX)^{-1}

Variance-covariance matrix of coefficients assuming heteroskedasticity (White, 1980),

Var(β) = (X'WX)^{-1}(X'Wdiag(e1^2, ..., en^2)WX)(X'WX)^{-1}

Finite sample correction factor for variance-coariance matrix of coefficients,

n/(n-k-1)

Tests,

Spoiler

Coefficient t-statistics,

t = b/se ~ t(df = n - 1)

Two sides tests use tables truncated at df = 100.

F-statistics,

F = ((SSRr - SSRu)/k) / ((SSRu)/(n-k-1)) ~ F(df1 = k, df2 = n - k - 1)
SSRu = E'WE
SSRr = (Y-l(l'w)^{-1}w'Y)'W(Y-l(l'w)^{-1}w'Y)
W = diag(runtime_1, ..., runtime_n) (WLS) or diag(1, ..., 1) (OLS)
w = (runtime_1, ..., runtime_n)' (WLS) or (1, ..., 1)' = l (OLS)
l = (1, ..., 1)'

Tests use tables truncated at df1 = 24 and df2 = 80.

Significance,

* p < 0.05
** p < 0.01
*** p < 0.001

Reproduction,

Quote

You can verify the script's coefficient estimates, standard errors, and statistics with the generated R code.

In R, OLS assuming homoskedasticity with finite sample corrections corresponds to,

ols <- lm(Y ~ X[, 2:5])
summary(ols)

Runtime WLS assuming homoskedasticity with finite sample corrections corresponds to,

wls <- lm(Y ~ X[, 2:5], weights = X[, 2])
summary(wls)

And OLS assuming heteroskedasticity (White, 1980) without finite sample corrections corresponds to,

library(lmtest)
library(sandwich)
coeftest(ols, vcov = vcovHC(ols, type = 'HC0'))

Design matrices are displayed as CSVs.

Discord

Spoiler

Discord for general discussion, questions/support, and proggies which I will add below,

https://discord.gg/SeJGVy8uYY

Support FAQ

Spoiler

I can't manually add an item?

Spoiler

There is probably no price data for that item for the pricing (latest, fiveMinute, oneHour, ...) that you've selected.

Here, I can add a Twisted bow using latest pricing, but since it has not been bought or sold in the last five minutes, I cannot add it with fiveMinute pricing.

The script isn't buying certain items?

Spoiler

You are probably trying to buy members items on a f2p world.

Here, I can buy Logs but not Revenant ether on a f2p world.

The generate / reset buttons in the GUI aren't working?

Spoiler

Your config parameter inputs are probably the wrong type or outside their support (refer to the config section above),

Here, I am getting NumberFormatExceptions because I am not inputting Java <int> types into the maxBidValue field. In the first case, I input a number larger than 2147483647, in the second case, I input a string of letters.

The script 'lost' (or 'made') astronomical amounts of GP off one or two trades!

Spoiler

Check the trade history for that item in the Trades CSV.

@dendabad12 V2.1 (members)

This huge 'loss' was actually caused by a double counting bug. Be sure to have a stable internet connection and not interupt the script when it is trying to collect trades.

This huge 'profit' was due to an implicit purchase price of zero as the item was already purchased/selling when I started the script.

 

 

Gallery of User Runs

Spoiler

@idhurtit  V2.0 (members)

@GoldenCurse V2.1 (members)

@gochimochi V2.1 (f2p)

@Kiwiszn V2.1 (members)

@damon3525 V2.1 (f2p)

@damon3525 V2.2 (members)

@oldboot V2.1 (members)

@50HZ V2.2 (members)

@khomey34 V2.2 (f2p)

@RandomKriss V2.2 (f2p/members)

@Markosa V2.2 (members)

@Sinky V2.3 (members)

@99sByAnyMeans V2.3 (members)

@khomey34 V2.3 (f2p)

@Daaamron V2.3 (members)

@RandomKriss V2.3 (f2p)

@? V2.3 (members)

@oldboot V2.3 (members)

@? [01] V2.3 (f2p)

@RandomKriss V2.3 (members)

@? [03] V2.3 (members)

@Daaamron  V2.3 (members)

@Daaamron V2.3 (members)

@SuperDad37611 [07] V2.3 (f2p)

@? [08] V2.3 (members)

@? [09] V2.3 (f2p)

@trevster120 [10] V2.4 (members)

@SuperDad37611 V2.4 (?)

@GoldenCurse V2.4 (members)

@GoldenCurse V2.4 (?)

@GoldenCurse V2.4 (members)

@oldboot V2.4 (members)

@trevster120 V2.4 (members)

@trevster120 V2.4 (members)

@trevster120 V2.4 (members)

@? [19] V2.4 (members)

@? [20] V2.4 (members)

@oldboot V2.4 (members)

@trevster120 V2.4 (members)

@? [23] V2.4 (members)

@? [23] V2.4 (f2p)

@khomey34 V2.4 (members)

@trevster120V2.4 (members)

@Asapfergg V2.4 (members)

@SuperDad37611 V2.4 (members)

 

Gallery of My Runs

Spoiler

V2.4 <New> On the Hour Discord Webhook Summary Notifications, More Items, Fewer Bugs

Spoiler

Profit::           N/A

Profit/Hour:  N/A

Trades CSV: N/A

V2.3: <New> Discord Webhook Notifications

Spoiler

Profit::           6,154,032

Profit/Hour:  1,113,339

Trades CSV: https://pastebin.com/7ET1PLE2

V2.2: <New> Regression Analysis

Spoiler

Profit::           1,063,992

Profit/Hour:  721,657

Trades CSV: https://pastebin.com/6i02BGV9

Profit::           1,630,856

Profit/Hour:  239,556

Trades CSV: https://pastebin.com/w3WQQcUy

V2.1: <New> Pricing Customisation, Post Session Inspection

Spoiler

Profit:           3,948,559

Profit/Hour: 228,681

Trades CSV: https://pastebin.com/52R3iABw

Profit:           1,312,080

Profit/Hour: 230,684

Trades CSV: https://pastebin.com/gvjWfb8b

Profit:            971,770   

Profit/Hour: 235,185

Trades CSV: https://pastebin.com/1HCnnCZE

V2.0: <New> Automatic Item Selection

Spoiler

Profit:           313,366   

Profit/Hour: 423,617

Trades CSV: https://pastebin.com/hZGeSdjf

V1.2: <New> Live Profit Graphing

Spoiler

Profit:           18,743   

Profit/Hour: 4,976

Trades CSV: https://pastebin.com/ZWhf3Xy5

Profit:            319,007   

Profit/Hour: 322,591

Trades CSV: https://pastebin.com/nkQ3Y4hb

V1.1: <New> Item selection

Spoiler

Profit:           57,406   

Profit/Hour: 82,270

Trades CSV: https://pastebin.com/UuhmakQ1

Profit:           210,412   

Profit/Hour: 195,077

Trades CSV: https://pastebin.com/Qe2gpZDu

V1.0: Only Logs and Death rune

Spoiler

Profit:           2,194   

Profit/Hour: 2,301

Trades CSV: https://pastebin.com/SP534wy0

Upload your proggies here or on Discord (with your DB username if you want to be credited) and I will add them above.

Please let me know if you encounter any bugs or would like any features added.

Edited March 17 by apnasus
my submission

[Hashtag 8694]

Released and promoted. Congratulations!

[SDN Bot 165]

humblePhlipper has been approved and is now live on the SDN!

Thanks!

[xyz111 81]

EPIC!

[brainbudt 4]

Unique script. Keep up the good work.

[CatnipandMilk 14]

very cool script. would be cool to select which items to flip.

[Gnar 22]

does the script use ge-tracker/runelite API or does it go off of current GE high low prices?

 

[YUNGTHUGFATGP 17]

Very cool

[SDN Bot 165]

humblePhlipper has been updated and is now live on the SDN!

It's currently at v1.1 and is available in the client!

Changes:

• Added support for all items
• Added QuickStart
• Added GUI
• Added Paint
• Added pause/resume bidding toggle
• Improved trade history accuracy

Time since request was made: 9 minutes, 48 seconds

Thanks!

[apnasus 49]

On 12/9/2023 at 5:43 AM, Gnar said:

does the script use ge-tracker/runelite API or does it go off of current GE high low prices?

 

Hey, it uses the OSRS Wiki API which was released about 2 years ago in partnership with Runelite, see https://oldschool.runescape.wiki/w/RuneScape:Real-time_Prices

This is a lot more accurate than Dreambot's internal API which proxies the above but with a delay of up to 20 minutes - this delay means it tends to lose money when I try using it

The script enters bids/asks at the latest low and high prices, respectively, which are updated every ~0.7 seconds or so. In the future I plan to develop more sophisticated pricing algorithms (with ARIMA, KF, ML etc.)