The Smart Shopping App
Challenges and solutions
Working with the android turned out to be a frustrating process from start to finish. Our first challenge was getting the project to work with Google’s Material Components, which took us several hours just to get a card to appear on the screen. Our biggest UI challenge was all learning how to use Android’s layout types and how to work with the verbose XML files. These issues arose merely due to our inexperience working with Android Studio and Material IO, but as the project went on our ability to translate designs to code greatly improved.
Due to 3 programmers working on this project at all times, there were several times where we would accidentally overwrite other programmers changes to the code base. This error came down to overtiredness from late nights and several programmers working on the same issues without realising. To mitigate this issue, we upped our git game and put a strong focus on using the Issue Driven Development software development methodology. This is where we would assign individual issues to a single programmer, which helped to keep each coder on task, and kept the merge conflicts at a minimum (Figure 1).
Figure 1. Issue tracking page. Task assignees are depicted by the initial on right.
Due to our app working with lists that are frequently updated programmatically, we found the best component for our purposes would be Android’s RecyclerView. This was a component that works well with dynamically inserted content. We used the recycler view to dynamically insert an instance of an XML layout where each item inserted represented an item on our list. The reason this became so stressful was due to all of us having to rote learn the component. The RecyclerView follows the MVC design pattern, where you create an adapter (controller) to take data from the model (in this case the user's list and our supermarket database), the adapter then represents this data on the View (the UI). However, the issue arose at the beginning where we attempted to write clean concise code so that we could write one adapter class and create different instances of it for all our different lists. This was fine in theory but ended up being so much more work as each list contained different UI components, but the adapter was only built to work with some. To fix this issue, we fully refactored the codebase and made a separate adapter class for each RecyclerView instance we created. A little verbose, but it got the job done nonetheless.
Another challenge of working with the estimotes was it required setting up the test area each time we wanted to test the code. Because our test area was a computer lab we sometimes could not use it as it was occupied by a class so it required some careful planning to test our app.
Testing final app
When we first tested our final app we found it was not performing as well as it was in previous tests. It was stuck in one aisle and it was difficult to change it to the other no matter where you were in the room. After repositioning the beacons slightly we were able to get the app working reasonably well. Some of the beacons were in slightly different position (< 20cm difference) during this first test which may have caused this problem although it was surprising that made such a big difference.
Once it was working we testing the app with the multilayer raw model by walking up and down the aisles and checking the aisle predicted. We found this model was still not performing as well as our testing suggested. It was generally good but it would often get stuck in the wrong aisle for extended periods. We decided to try some other models such as Bayes raw, Bayes non-sliding median and multilayer perceptron non-sliding median. Of these, we found Bayes non-sliding median appeared to perform the best. With this model, the app would predict the aisle correctly most of the time and seldom get stuck in the wrong aisle for extended periods of time. One issue with this model was that it would frequently flicker into the wrong aisle for short periods of times.
The fact that multilayer perceptron did not work out as well as we thought it would, highlights the importance of making sure results are replicable. When running our tests for the previous blog, we could have run two or three independent tests to make sure the results could be repeated. Also, we could have prolonged the test time to five minutes instead of two minutes to get a better sample size.
Future developments
One of the main things that could improve our app is to reduce the amount of flicker between the aisles while predicting. We might be able to achieve this by simply adding extra training data over hot spots where the app tended to flicker (Figure 2). In general, more data is likely to help improve the performance of our models. We could train the models for an hour each aisle and see if this improves them.
Figure 2. Problem areas for the two best models.
Another simple approach to decrease flickering is to only change the aisle predicted when two successive readings have been received. This could reduce random flickers but may introduce other issues such as a long lag time when changing aisles.
A major way we could improve our app is to subdivide the aisles into smaller sections. We could train it in a similar way by walking only in the subsections before training a more complex model. The app would better approximate our location and a will produce a more concise list of items at the different locations.
Natural user interface
A natural user interface is an interface that is effectively invisible to the user. We have incorporated elements of natural UI through the design of our user interface and the main concept. For example, our application allows the user to walk around the supermarket with the application open (just like how they would walk around with a shopping list) and the only requirement is to have Bluetooth turned on. There is no need for the user to know how to the application works, they just move into the new aisle and their list is updated for them. The design elements that incorporate natural user interface is the idea of the list and how we update it based on the users interaction. We have designed the application to allow the user to tick a checkbox once they have the item in their cart. This will then create a strike through of the item, therefore linking back to a natural list.
Ubiquitous computing
Ultimately, we ended up with a complete prototype for a ubiquitous shopping app. It has many UI features that give the app a clean look and improves usability. What makes it ubiquitous is that it automatically updates the aisle based on the signal strength from embedded sensors, which subsequently tracks your position in the room. The hope is that our user will simply stroll through the supermarket picking up items as they appear up on the screen without having to think about the underlying technology. This app will hopefully improve the user’s experience by removing some of the burden of the ‘warehousing’ while shopping, freeing them to think about other things, like what to cook for dinner.
Working with the android turned out to be a frustrating process from start to finish. Our first challenge was getting the project to work with Google’s Material Components, which took us several hours just to get a card to appear on the screen. Our biggest UI challenge was all learning how to use Android’s layout types and how to work with the verbose XML files. These issues arose merely due to our inexperience working with Android Studio and Material IO, but as the project went on our ability to translate designs to code greatly improved.
Due to 3 programmers working on this project at all times, there were several times where we would accidentally overwrite other programmers changes to the code base. This error came down to overtiredness from late nights and several programmers working on the same issues without realising. To mitigate this issue, we upped our git game and put a strong focus on using the Issue Driven Development software development methodology. This is where we would assign individual issues to a single programmer, which helped to keep each coder on task, and kept the merge conflicts at a minimum (Figure 1).
Figure 1. Issue tracking page. Task assignees are depicted by the initial on right.
Due to our app working with lists that are frequently updated programmatically, we found the best component for our purposes would be Android’s RecyclerView. This was a component that works well with dynamically inserted content. We used the recycler view to dynamically insert an instance of an XML layout where each item inserted represented an item on our list. The reason this became so stressful was due to all of us having to rote learn the component. The RecyclerView follows the MVC design pattern, where you create an adapter (controller) to take data from the model (in this case the user's list and our supermarket database), the adapter then represents this data on the View (the UI). However, the issue arose at the beginning where we attempted to write clean concise code so that we could write one adapter class and create different instances of it for all our different lists. This was fine in theory but ended up being so much more work as each list contained different UI components, but the adapter was only built to work with some. To fix this issue, we fully refactored the codebase and made a separate adapter class for each RecyclerView instance we created. A little verbose, but it got the job done nonetheless.
Another challenge of working with the estimotes was it required setting up the test area each time we wanted to test the code. Because our test area was a computer lab we sometimes could not use it as it was occupied by a class so it required some careful planning to test our app.
Testing final app
When we first tested our final app we found it was not performing as well as it was in previous tests. It was stuck in one aisle and it was difficult to change it to the other no matter where you were in the room. After repositioning the beacons slightly we were able to get the app working reasonably well. Some of the beacons were in slightly different position (< 20cm difference) during this first test which may have caused this problem although it was surprising that made such a big difference.
Once it was working we testing the app with the multilayer raw model by walking up and down the aisles and checking the aisle predicted. We found this model was still not performing as well as our testing suggested. It was generally good but it would often get stuck in the wrong aisle for extended periods. We decided to try some other models such as Bayes raw, Bayes non-sliding median and multilayer perceptron non-sliding median. Of these, we found Bayes non-sliding median appeared to perform the best. With this model, the app would predict the aisle correctly most of the time and seldom get stuck in the wrong aisle for extended periods of time. One issue with this model was that it would frequently flicker into the wrong aisle for short periods of times.
The fact that multilayer perceptron did not work out as well as we thought it would, highlights the importance of making sure results are replicable. When running our tests for the previous blog, we could have run two or three independent tests to make sure the results could be repeated. Also, we could have prolonged the test time to five minutes instead of two minutes to get a better sample size.
Future developments
One of the main things that could improve our app is to reduce the amount of flicker between the aisles while predicting. We might be able to achieve this by simply adding extra training data over hot spots where the app tended to flicker (Figure 2). In general, more data is likely to help improve the performance of our models. We could train the models for an hour each aisle and see if this improves them.
Figure 2. Problem areas for the two best models.
Another simple approach to decrease flickering is to only change the aisle predicted when two successive readings have been received. This could reduce random flickers but may introduce other issues such as a long lag time when changing aisles.
A major way we could improve our app is to subdivide the aisles into smaller sections. We could train it in a similar way by walking only in the subsections before training a more complex model. The app would better approximate our location and a will produce a more concise list of items at the different locations.
Natural user interface
A natural user interface is an interface that is effectively invisible to the user. We have incorporated elements of natural UI through the design of our user interface and the main concept. For example, our application allows the user to walk around the supermarket with the application open (just like how they would walk around with a shopping list) and the only requirement is to have Bluetooth turned on. There is no need for the user to know how to the application works, they just move into the new aisle and their list is updated for them. The design elements that incorporate natural user interface is the idea of the list and how we update it based on the users interaction. We have designed the application to allow the user to tick a checkbox once they have the item in their cart. This will then create a strike through of the item, therefore linking back to a natural list.
Ubiquitous computing
Ultimately, we ended up with a complete prototype for a ubiquitous shopping app. It has many UI features that give the app a clean look and improves usability. What makes it ubiquitous is that it automatically updates the aisle based on the signal strength from embedded sensors, which subsequently tracks your position in the room. The hope is that our user will simply stroll through the supermarket picking up items as they appear up on the screen without having to think about the underlying technology. This app will hopefully improve the user’s experience by removing some of the burden of the ‘warehousing’ while shopping, freeing them to think about other things, like what to cook for dinner.
Comments
Post a Comment