Computation Offloading for Efficient Execution of AI applications in Edge Computing

With the dawn of powerful and energy-efficient hardware in the low-power computing space, there is the need to decide to go either for performance or for efficiency. Although not completely exclusive to each other there is usually a winner during practical applications. However, the chosen solution might not really work due to energy constraints set by the limited power sources of mobile devices. The performance capabilities of these edge devices are severely limited to thermal and energy constraints. For the sake of improving efficiency and accuracy in machine learning and running neural networks onboard such portable devices we are faced with the following simple yet brutal choice. The first option involves wasting local resources in the form of energy and computational power for a low-accuracy output. The second option is sending (offloading) the job to be processed onto a remote server with high accuracy but incurring massive energy penalties on the local level in the form of energy wasted for transmission and at the server in the form of the energy used to compute the result. The idea of this research project is to develop an algorithm that can help us make the optimum decision to offload the task from an edge device to a server so that we can use less energy and increase average accuracy across all jobs. We have used two devices over a wireless [WLAN] network. The first device is a low power(6.5W) Raspberry pi 4 which represents our edge device It is running a small version of the MobileNetV2 image classifier. The second device is a computer as a server which is high power(120W) and it is running a larger version of MobileNet V3 image classifier which is more accurate. We are measuring the energy consumption of both devices with external measuring devices. The Pi has a set of images(jobs) on its local memory that it has to classify. Before each image can be classified it must be reshaped into 224x224 pixels. We have an energy budget that is the sum of energy required to reshape all jobs and classify all jobs on the local device. The local device has three options for each image, it can either reshape and classify itself paying the associated energy costs, it can transmit to the server for reshaping and classification by paying the transmission cost and finally it can reshape the image and then transmit to pay the reshape energy cost to save on transmission energy costs for larger files. Our goal is to somehow make the best decision out of the three to finally consume less energy than our energy budget while simultaneously improving accuracy. This decision is made based upon the expected values of these different energies for each image, using lookup tables that have been created by data collection, essentially making it a computing time(energy) to space trade-off. The Implemented solution was run on three sets of data from the ImageNet database and it allows us to improve upon the energy budget by 1.93 % in our best case and an accuracy gain of .42 % in top 1 accuracy. The other two sets that have been tried stand to gain .36 %in accuracy and 1.26 % in energy budget for set 2 and .22 % for accuracy and .89 %in energy budget in set 3. Since there is'nt much focus on research in the field of studying energy consumption for networks and edge devices these results reveal to us the possibility of squeezing water out of rocks when it comes to saving energy. If the data is favourable and distributed in a certain size range, more than 5-10 % of energy savings can be made.