Jump to content

Welcome to DreamBot

VIP Enhancement

Want to upgrade your DreamBot experience? Consider signing up for VIP!
VIP allows you to run as many accounts as you want, view the forums ad-free, receive 10% off all script purchases, and so much more!
Visit the store to learn about all of these great features!

Upgrade to VIP Now
Frequently Asked Questions
  • Are you not able to open the client? Make sure you have Java 8 installed
  • Help! My bot doesn't do anything! Enable fresh start in client settings and restart the client
  • How to purchase with PayPal/OSRS gold? You can purchase vouchers from other users
  • Try asking for help in the chatbox
Download the DreamBot client today!
beezdul

Neural Network Mouse Movement

Recommended Posts

Video

This is a small project I'm working on where I use a neural network to create mouse movements. In theory, the movements themselves should be completely undetectable (and the results are pretty impressive and look almost exactly like my real movements). Currently I've trained it on about 4000 clicks from my own data.

 

How it works:

The network is a 3 layer fully connected network. The input is 6 float values (random value 0.000-0.001,last start delta X,last start delta Y,current delta X,current delta Y,1.0)

The network has a 1000 neuron layer connected to a 400 neuron layer (trained with 50% dropout), and the activation function is leaky relu (tf.nn.leaky_relu).

The outputs are 456 values, each ranging from -1 to 1. The output is grouped into 152 sets of 3, being the delta x, delta y, and either 1 or -1 (-1 signals that the movement is complete).

The deltas are an amount from the end position, divided by 1000. (So if the destination is at 1000,500 and our mouse is at 0,0, then the delta is 1.0,0.5). The delta will always decrease to 0,0 as that is the pixel we're moving to. It's easier for the network to learn to decay to 0,0 from a starting delta, than to go from 0,0 to an end point. Now there's still a bit of jerkiness on the first position, most likely due to not enough data, so I also apply a small amount of smoothing to the points to reduce it.

I'm using tensorflow (python) for the network.

 

Gathering data:

I made a simple program that records all mouse movements. When the square is clicked, it uses the destination as the end point, and then re-creates the nice delta data from there, then resets and moves the square to another random position. The reason why the square is so large is because it leads to more fluid and free mouse movements.

image.png.4c58a74961e694aa89d61afe6783127a.png

 

Plans:

Once I get things smoothed out I'll try to implement it in some of my private scripts, and then open-source it. I'm also currently testing how it affects ban rates on brand new F2P bots, compared to a control set.

Share this post


Link to post
Share on other sites
On 6/30/2019 at 12:37 PM, yeeter01 said:

What is the performance hit on implementing something such as this tho?

Pretty much 0

Edit: Tested to be around 0.5-1ms per path generated.

Edited by beezdul

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...