#### VIP Enhancement

VIP allows you to run as many accounts as you want, view the forums ad-free, receive 10% off all script purchases, and so much more!
Visit the store to learn about all of these great features!

• Are you not able to open the client? Make sure you have Java 8 installed
• Help! My bot doesn't do anything! Enable fresh start in client settings and restart the client
• How to purchase with PayPal/OSRS gold? You can purchase vouchers from other users
• Try asking for help in the chatbox

# Neural Network Mouse Movement

## Recommended Posts

This is a small project I'm working on where I use a neural network to create mouse movements. In theory, the movements themselves should be completely undetectable (and the results are pretty impressive and look almost exactly like my real movements). Currently I've trained it on about 4000 clicks from my own data.

How it works:

The network is a 3 layer fully connected network. The input is 6 float values (random value 0.000-0.001,last start delta X,last start delta Y,current delta X,current delta Y,1.0)

The network has a 1000 neuron layer connected to a 400 neuron layer (trained with 50% dropout), and the activation function is leaky relu (tf.nn.leaky_relu).

The outputs are 456 values, each ranging from -1 to 1. The output is grouped into 152 sets of 3, being the delta x, delta y, and either 1 or -1 (-1 signals that the movement is complete).

The deltas are an amount from the end position, divided by 1000. (So if the destination is at 1000,500 and our mouse is at 0,0, then the delta is 1.0,0.5). The delta will always decrease to 0,0 as that is the pixel we're moving to. It's easier for the network to learn to decay to 0,0 from a starting delta, than to go from 0,0 to an end point. Now there's still a bit of jerkiness on the first position, most likely due to not enough data, so I also apply a small amount of smoothing to the points to reduce it.

I'm using tensorflow (python) for the network.

Gathering data:

I made a simple program that records all mouse movements. When the square is clicked, it uses the destination as the end point, and then re-creates the nice delta data from there, then resets and moves the square to another random position. The reason why the square is so large is because it leads to more fluid and free mouse movements.

Plans:

Once I get things smoothed out I'll try to implement it in some of my private scripts, and then open-source it. I'm also currently testing how it affects ban rates on brand new F2P bots, compared to a control set.

##### Share on other sites

Neat af, looking for further updates

##### Share on other sites

What is the performance hit on implementing something such as this tho?

##### Share on other sites
On 6/30/2019 at 12:37 PM, yeeter01 said:

What is the performance hit on implementing something such as this tho?

Pretty much 0

Edit: Tested to be around 0.5-1ms per path generated.

Edited by beezdul

##### Share on other sites

Hey, I have a few questions regarding this if you have a few minutes to spare. I sent you a friend request on discord (Edaredan).

##### Share on other sites
6 hours ago, Mimzy said:

Hey, I have a few questions regarding this if you have a few minutes to spare. I sent you a friend request on discord (Edaredan).

I just added you on discord.

##### Share on other sites

Hi I have added you on discord as well. I'm in the beginning stages of using a NN to generate human mouse movements. Eventually I want to get use GANS to add variation to it.

##### Share on other sites

Hi, this sounds great! Do you already have a GitHub where we could look at it?

## Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

×   Pasted as rich text.   Paste as plain text instead

Only 75 emoji are allowed.