Moving on to testing: Precision vs Accuracy

Set Up

Coming into our second week of working with the estimote beacons, we met up after the Monday lecture to discuss what we achieved in our first week and what our next steps are moving forward. Last week we all downloaded android studio to our laptops and forked from a central repository. Oliver then coded a small android application which downloaded and tracked information from the closest beacon.

In our meeting we discussed a few variables that we could change in order to test for precision and accuracy of the sensors, including changing the distance of the beacons, the orientation (whether the beacon is flat on the surface or stuck onto the wall) and the surrounding environment of the room (including size and whether magnetism would affect the sensors).

Tuesday’s meeting consisted of us tidying up the code to our application, enabling us to combine all the information from multiple beacons, not just the closest one. It also gave us a chance to talk to Tobi and realise we were probably thinking too big to start and that we should focus on getting the six sensors to work in a single space before we even think about comparing to other areas. With this in mind we measured out our space and came up with the following diagram:

The idea here is that we decided to go with three sensors on opposite walls as the set up, mainly because Tobias and Jacob used a different setup and we wanted to see if this way would produce better results. Sensor positions were decided by width of the room/4 so that they were evenly spaced. We decided that we would use a 1X1m grid for testing purposes where we plan on testing the coordinates using our own coordinate space.

The beacon information for set up is:

Beacon Information
7d0209318d9fec53f1add605f96af12e, Wall 1, 176 cm to left corner
fea6c067c3af3359cc56d94b1c79a339, Wall 1, 352 cm to left corner
25120c4f984555d741bda6329b659417, Wall 1, 176 cm to right corner
519433caa42e2e5717a7369019f2000d, Wall 3, 176 cm to left corner
539a0146ce4fe1f880519b346c9aca29, Wall 3, 352 cm to left corner
ced11210d9981fe945f394a35c4e9203, Wall 3, 176 cm to right corner

Precision vs Accuracy

The next step is to make sure our app for getting the coordinate is working using a mesh and then we can mark out the room with our 1X1 grid and get to testing. Before we can define our method we need to look into what accuracy and precision is so that we can properly meet the aim of the first milestone.

Precision:

Precision is about the repeatability of the measured values, ie; how close each measured value is to each other measured value. For example, we would expect that if our sensors have high precision that if we placed a phone in the exact same position multiple times we would receive the same sensor reading or similar each time.  This would be relatively easy to measure by simply recording the position and comparing the values over a set amount of trials. 

Accuracy:

Accuracy refers to how close a measured value is to a known value. For example, when tracking a bluetooth device with our sensors, the sensors accuracy would be determined by how close their reported location is to the real "location" of the device. Already this brings to light a question about how we first find the known value (in this case, the known location) so that we have a value to compare the sensor readings to. 

Proposed Method:

• Mark out 1 X 1 grid with tape marks on the floor of the room (where possible)
• Place phone on intersection and leave until readings settle
• Record estimote coordinate position and actual position
• Repeat for all intersections until grid has been completed, these results can then be used to say something about the accuracy 
• Repeat entire process X amount of times so that we can make conclusions about precision.

Note:
We have chosen to map a room with a 1m by 1m grid which will allow us to do some cool visualisations of the beacons accuracy using colour coding. The colour coding could just work by colouring a square green if the measurement was accurate at that grid point, and red if it was not. This allows us to better visualise the accuracy of the beacons all over a grid, so if we start to notice a pattern of the colour red around a certain beacon’s location, then we can start to investigate that specific beacon and the issues that may be affecting readings.