Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Intelligent Processing of Data Streams on the Edge Using Reinforcement Learning
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences.
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences.ORCID iD: 0000-0002-6617-8683
2023 (English)In: 2023 IEEE International Conference on Communications Workshops (ICC Workshops), IEEE (Institute of Electrical and Electronics Engineers) , 2023Conference paper, Published paper (Refereed)
Abstract [en]

A key challenge in many IoT applications is to en-sure energy efficiency while processing large amounts of streaming data at the edge. Nodes often need to process time-sensitive data using limited computing and communication resources. To that end, we design a novel R - Learning based Offloading framework, RLO, that allows edge nodes to learn energy optimal decisions from experience regarding processing incoming data streams. In particular, when should the node process data locally? When should it transmit data to be processed by a fog node? And when should it store data for later processing? We validate our results on both real and simulated data streams. Simulation results show that RLO learns with time to achieve better overall-rewards with respect to three existing baseline schemes. Moreover, the proposed algorithm excels the existing baseline schemes when different priorities were set on the two objectives. We also illustrate how to adjust the priorities of the two objectives based on the application requirements and network constraints.

Place, publisher, year, edition, pages
IEEE (Institute of Electrical and Electronics Engineers) , 2023.
Series
IEEE International Conference on Communications workshops, ISSN 2164-7038, E-ISSN 2694-2941
Keywords [en]
Reinforcement learning, Data stream processing, Offloading, Edge, IoT
National Category
Computer Sciences
Research subject
Computer and Systems Sciences
Identifiers
URN: urn:nbn:se:su:diva-223319DOI: 10.1109/ICCWorkshops57953.2023.10283692ISI: 001094861300207Scopus ID: 2-s2.0-85177821473ISBN: 979-8-3503-3308-4 (print)OAI: oai:DiVA.org:su-223319DiVA, id: diva2:1807312
Conference
IEEE ICC 2023 Workshop on Scalable and Trustworthy AI for 6G Wireless Networks (6GSTRAIN)
Available from: 2023-10-25 Created: 2023-10-25 Last updated: 2026-04-13
In thesis
1. AI-Driven Multi-objective Decision-Making With Applications to IoT
Open this publication in new window or tab >>AI-Driven Multi-objective Decision-Making With Applications to IoT
2026 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Artificial intelligence (AI) plays an increasingly central role in enabling autonomous decision-making in complex, uncertain environments. Many modern systems must optimise multiple, often conflicting objectives while operating under dynamic resource constraints and incomplete knowledge of system dynamics. Classical approaches such as dynamic programming, constrained stochastic optimisation, and static multi-objective scalarisation provide principled solutions when accurate models are available. However, in distributed and stochastic environments such as the Internet of Things (IoT), system dynamics are often unknown, non-stationary, and resource-limited, making purely model-based methods difficult to apply.

Reinforcement learning (RL) and online learning offer an alternative by enabling policy adaptation through interaction rather than relying on explicit system models. Within multi-objective settings, existing approaches often assume fixed scalarisation weights or externally specified preferences and typically focus on learning policies for given trade-offs. In dynamic IoT systems, however, both resource constraints and preference parameters may vary over time, requiring algorithms that can adapt efficiently without repeated retraining or centralised coordination.

This thesis investigates how AI-based methods, with a primary focus on reinforcement learning and complementary distributed learning techniques, can support adaptive multi-objective decision-making under communication constraints, explicit resource limitations, and dynamically changing trade-offs. The research is organised around three themes. First, communication-efficient distributed learning methods are developed to balance model accuracy and communication cost infederated learning through adaptive sparsification. Second, constrained bandit and reinforcement learning formulations are proposed to incorporate explicit and time-varying resource constraints while maintaining theoretical performance guarantees. Third, multi-objective reinforcement learning methods are designed to adapt routing decisions indistributed IoT systems under dynamically changing energy–reliabilitytrade-offs without retraining.

Overall, the thesis demonstrates that integrating communication awareness, constraint handling, and preference adaptation directly into learning algorithms is essential for reliable AI-based decision-makingin IoT environments. The results provide both algorithmic advances and a conceptual framework for designing autonomous systems that operate robustly under dynamic objectives and limited resources.

Abstract [sv]

Artificiell intelligens (AI) spelar en allt viktigare roll för att möjliggöra autonomt beslutsfattande i komplexa och osäkra miljöer. Många moderna system måste optimera flera, ofta motstridiga mål samtidigt som de verkar under dynamiska resursbegränsningar och med ofullständig kunskap om systemets dynamik. Klassiska metoder såsom dynamisk programmering, begränsad stokastisk optimering och statisk multiobjektiv skalärisering erbjuder principiella lösningar när exakta modeller finns tillgängliga. I distribuerade och stokastiska miljöer, såsom Internet of Things (IoT), är dock systemdynamiken ofta okänd, icke-stationär och resursbegränsad, vilket gör rent modellbaserade metoder svåra att tillämpa.

Förstärkningsinlärning (reinforcement learning, RL) och onlineinlärning erbjuder ett alternativ genom att möjliggöra policyanpassning genom interaktionsnarare än genom explicita systemmodeller. Inom multiobjektiva problemantar befintliga metoder ofta fasta skaläriseringsvikter eller externt specificerade preferenser och fokuserar främst på att lära policyer för givna avvägningar.I dynamiska IoT-system kan dock både resursbegränsningar och preferensparametrarförändras över tid, vilket kräver algoritmer som kan anpassa sig effektivtutan upprepad ominlärning eller centraliserad styrning.

Denna avhandling undersöker hur AI-baserade metoder, med särskilt fokus på förstärkningsinlärning och kompletterande distribuerade inlärningsmetoder,kan stödja adaptivt multiobjektivt beslutsfattande under kommunikationsbegränsningar,explicita resurskrav och dynamiskt föränderliga avvägningar.Forskningen är organiserad kring tre huvudteman. För det första utveckla kommunikationseffektiva distribuerade inlärningsmetoder som balanserar modellnoggrannhet och kommunikationskostnad i federerad inlärning genom adaptiv sparsifiering. För det andra föreslås begränsade bandit- och förstärkningsinlärningsformuleringar för att hantera explicita och tidsvarierande resursbegränsningar med teoretiska prestandagarantier. För det tredje utvecklasmultiobjektiv förstärkningsinlärning för att möjliggöra adaptiv routingi distribuerade IoT-system under dynamiskt föränderliga avvägningar mellanenergiförbrukning och tillförlitlighet, utan behov av ominlärning.

Sammanfattningsvis visar avhandlingen att det är avgörande att explicit integrera kommunikationsmedvetenhet, begränsningshantering och preferensanpassning i inlärningsalgoritmer för att uppnå tillförlitligt AI-baserat beslutsfattande i IoT-miljöer. Resultaten bidrar med både algoritmiska framsteg och konceptuell grund för att utforma autonoma system som kan verka robusta under dynamiska mål och begränsade resurser.

Place, publisher, year, edition, pages
Stockholm: Department of Computer and Systems Sciences, Stockholm University, 2026. p. 116
Series
Report Series / Department of Computer & Systems Sciences, ISSN 1101-8526 ; 26-005
Keywords
Artificial Intelligence, Multi-objective, Internet of Things, Federated Learning, Reinforcement Learning
National Category
Communication Systems Computer Systems Telecommunications Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Computer and Systems Sciences
Identifiers
urn:nbn:se:su:diva-254165 (URN)978-91-8107-602-8 (ISBN)978-91-8107-603-5 (ISBN)
Public defence
2026-05-29, Small Auditorium NOD (Lilla hörsalen), plan 2, Borgarfjordsgatan 12, Kista, 13:00 (English)
Opponent
Supervisors
Available from: 2026-05-06 Created: 2026-04-13 Last updated: 2026-04-28Bibliographically approved

Open Access in DiVA

fulltext(678 kB)178 downloads
File information
File name FULLTEXT01.pdfFile size 678 kBChecksum SHA-512
f0f17588a039206eb56957ea39df7b3b6396078ec32fd0813e399957360e765445a8e6eb3f262bde7127719cd8f0861a3a3bd1e06a876084e95d5467c69fa3d6
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Vaishnav, ShubhamMagnússon, Sindri
By organisation
Department of Computer and Systems Sciences
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 179 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 387 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf