2026-06-27T23:23:44Zhttps://docta.ucm.es/rest/oai/request
oai:docta.ucm.es:20.500.14352/964842025-05-09T23:47:11Zcom_20.500.14352_14col_20.500.14352_15
Docta Complutense author Huang, Darong author Pahlevan, Ali author Costero Valero, Luis María author Zapater, Marina author Atienza Alonso, David 2024-01-30T12:29:21Z 2024-01-30T12:29:21Z 2022 D. Huang, A. Pahlevan, L. Costero, M. Zapater and D. Atienza, "Reinforcement Learning-Based Joint Reliability and Performance Optimization for Hybrid-Cache Computing Servers," in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 41, no. 12, pp. 5596-5609, Dec. 2022, doi: 10.1109/TCAD.2022.3158832. 1937-4151 10.1109/TCAD.2022.3158832 https://hdl.handle.net/20.500.14352/96484 https://doi.org/10.1109/TCAD.2022.3158832 Computing servers play a key role in the development and process of emerging compute-intensive applications in recent years. However, they need to operate efficiently from an energy perspective viewpoint, while maximizing the performance and lifetime of the hottest server components (i.e., cores and cache). Previous methods focused on either improving energy efficiency by adopting new hybrid-cache architectures including the resistive random-access memory (RRAM) and static random-access memory (SRAM) at the hardware level, or exploring tradeoffs between lifetime limitation and performance of multicore processors under stable workloads conditions. Therefore, no work has so far proposed a co-optimization method with hybrid-cache-based server architectures for real-life dynamic scenarios taking into account scalability, performance, lifetime reliability, and energy efficiency at the same time. In this article, we first formulate a reliability model for the hybrid-cache architecture to enable precise lifetime reliability management and energy efficiency optimization. We also include the performance and energy overheads of cache switching, and optimize the benefits of hybrid-cache usage for better energy efficiency and performance. Then, we propose a runtime Q-learning-based reliability management and performance optimization approach for multicore microprocessors with the hybrid-cache architecture, jointly incorporated with a dynamic preemptive priority queue management method to improve the overall tasks’ performance by targeting to respect their end time limits. Experimental results show that our proposed method achieves up to 44% average performance (i.e., tasks execution time) improvement, while maintaining the whole system design lifetime longer than five years, when compared to the latest state-of-the-art energy efficiency optimization and reliability management methods for computing servers. eng Reinforcement Learning-Based Joint Reliability and Performance Optimization for Hybrid-Cache Computing Servers journal article UGFyYSBxdWUgZWwgcmVwb3NpdG9yaW8gaW5zdGl0dWNpb25hbCBkZSBsYSBVbml2ZXJzaWRhZCBkZSBNYWRyaWQgcHVlZGEgYWxtYWNlbmFyIHkgZGlzdHJpYnVpciBlbCBvYmpldG8gZGlnaXRhbCBkZXBvc2l0YWRvLCBlcyBuZWNlc2FyaW8gcXVlIGxhIHBlcnNvbmEgcXVlIGhhZ2EgZWwgZGVww7NzaXRvIGxlYSB5IGFjZXB0ZSBsYXMgY29uZGljaW9uZXMgZXN0YWJsZWNpZGFzIGVuIGVzdGEgbGljZW5jaWE6CuKAoiBBY2VwdGFuZG8gZXN0YSBsaWNlbmNpYSwgdXN0ZWQgY29tbyBhdXRvci9yZXMgeS9vIHByb3BpZXRhcmlvL3MgZGUgbG9zIGRlcmVjaG9zIGRlIHByb3BpZWRhZCBpbnRlbGVjdHVhbCwgbyBsYSBwZXJzb25hIGRlbGVnYWRhIHBhcmEgaGFjZXJsbyAsIG90b3JnYSBhbCByZXBvc2l0b3JpbyBkZSBsYSBVbml2ZXJzaWRhZCBDb21wbHV0ZW5zZSBkZSBNYWRyaWQgZWwgZGVyZWNobyBubyBleGNsdXNpdm8gZGUgcmVwcm9kdWNjacOzbiB5IGRpc3RyaWJ1Y2nDs24gZGUgc3Ugb2JyYSBlbiBjdWFscXVpZXIgZm9ybWF0byB5IG1lZGlvLgrigKIgRWwgZGVwb3NpdGFudGUsIGVuIGNhc28gZGUgdW5hIG9icmEgY29uIG3DoXMgZGUgdW4gYXV0b3IsIGdhcmFudGl6YSBxdWUgbG8gaGFjZSByZXNwb25zYWJsZW1lbnRlIGVuIG5vbWJyZSB5IGNvbiBjb25zZW50aW1pZW50byBkZSBsb3MgZGVtw6FzIGF1dG9yZXMuCuKAoiBBY2VwdGEgcXVlIGVsIHJlcG9zaXRvcmlvIGRlIGxhIFVuaXZlcnNpZGFkIENvbXBsdXRlbnNlIGRlIE1hZHJpZCBwdWVkYSBjb25zZXJ2YXIgbcOhcyBkZSB1bmEgY29waWEgZGUgZXN0ZSBkb2N1bWVudG8geSwgc2luIGFsdGVyYXIgc3UgY29udGVuaWRvLCBjb252ZXJ0aXJsbyBhIGN1YWxxdWllciBmb3JtYXRvIGRlIGZpY2hlcm8sIG1lZGlvIG8gc29wb3J0ZSwgcGFyYSBwcm9ww7NzaXRvcyBkZSBzZWd1cmlkYWQgeSBwcmVzZXJ2YWNpw7NuLgrigKIgRGVjbGFyYSBxdWUgZWwgZG9jdW1lbnRvIGVzIHVuIHRyYWJham8gb3JpZ2luYWwgc3V5bywgbyBxdWUgZXMgbGEgcGVyc29uYSBkZWxlZ2FkYSBwYXJhIHJlYWxpemFyIGVsIGRlcMOzc2l0byB5LCBwb3IgdGFudG8sIGN1ZW50YSBjb24gbGEgcG90ZXN0YWQgcGFyYSBvdG9yZ2FyIGxvcyBkZXJlY2hvcyBjb250ZW5pZG9zIGVuIGVzdGEgbGljZW5jaWEuIFRhbWJpw6luIGRlY2xhcmEgcXVlIHN1IGRvY3VtZW50byBubyBpbmZyaW5nZSBsbyBlc3RhYmxlY2lkbyBlbiBsYSB2aWdlbnRlIGxlZ2lzbGFjacOzbiBkZSBwcm9waWVkYWQgaW50ZWxlY3R1YWwuCuKAoiBTaSBlbCBkb2N1bWVudG8gY29udGllbmUgbWF0ZXJpYWxlcyBkZSBsb3MgY3VhbGVzIG5vIHRpZW5lIGxvcyBkZXJlY2hvcyBkZSBhdXRvciwgZGVjbGFyYSBxdWUgaGEgb2J0ZW5pZG8gZWwgcGVybWlzbyBzaW4gcmVzdHJpY2Npw7NuIGRlbCBwcm9waWV0YXJpbyBwYXJhIG90b3JnYXIgYSBsYSBVbml2ZXJzaWRhZCBDb21wbHV0ZW5zZSBkZSBNYWRyaWQgbG9zIGRlcmVjaG9zIHJlcXVlcmlkb3MgcG9yIGVzdGEgbGljZW5jaWEsIHkgcXVlIGVzZSBtYXRlcmlhbCwgY3V5b3MgZGVyZWNob3Mgc29uIGRlIHRlcmNlcm9zLCBlc3TDoSBjbGFyYW1lbnRlIGlkZW50aWZpY2FkbyB5IHJlY29ub2NpZG8gZGVudHJvIGRlbCBjb250ZW5pZG8gZGVsIGRvY3VtZW50by4K4oCiIFNpIGVsIGRvY3VtZW50byBzZSBiYXNhIGVuIHVuYSBvYnJhIHF1ZSBoYSBzaWRvIHBhdHJvY2luYWRhIG8gYXBveWFkYSBwb3IgdW5hIGFnZW5jaWEgdSBvcmdhbml6YWNpw7NuIGRpZmVyZW50ZSBkZSBsYSBVbml2ZXJzaWRhZCBDb21wbHV0ZW5zZSBkZSBNYWRyaWQsIHVzdGVkIGRlY2xhcmEgcXVlIGhhIGN1bXBsaWRvIGNvbiBsYXMgb2JsaWdhY2lvbmVzIHJlcXVlcmlkYXMgcG9yIGVzdGUgY29udHJhdG8gbyBhY3VlcmRvLgrigKIgTGEgVW5pdmVyc2lkYWQgQ29tcGx1dGVuc2UgZGUgTWFkcmlkIGlkZW50aWZpY2Fyw6EgY2xhcmFtZW50ZSBzdS9zIG5vbWJyZS9zIGNvbW8gZWwvbG9zIGF1dG9yL2VzIG8gcHJvcGlldGFyaW8vcyBkZSBsb3MgZGVyZWNob3MgZGVsIGRvY3VtZW50bywgeSBubyBoYXLDoSBuaW5ndW5hIGFsdGVyYWNpw7NuIGRlIHN1IGRvY3VtZW50byBkaWZlcmVudGUgYSBsYXMgcGVybWl0aWRhcyBlbiBlc3RhIGxpY2VuY2lhLgrigKIgVHJhYmFqb3MgZGVwb3NpdGFkb3MgcG9yIHN1IHByb3BpbyBhdXRvcjogYWwgYXV0by1hcmNoaXZhciBlc3RhIGNvbGVjY2nDs24gZGUgZmljaGVyb3MgeSBtZXRhZGF0b3MgYmlibGlvZ3LDoWZpY29zIGFzb2NpYWRvcywgb3RvcmdvIGFsIHJlcG9zaXRvcmlvIGluc3RpdHVjaW9uYWwgZGUgbGEgVW5pdmVyc2lkYWQgQ29tcGx1dGVuc2UgZGUgTWFkcmlkIGVsIGRlcmVjaG8gcGFyYSBhbG1hY2VuYXJsb3MgeSBkaXN0cmlidWlybG9zLiBEZWNsYXJvIHF1ZSBlc3RlIG1hdGVyaWFsIGVzIGRlIG1pIHByb3BpZWRhZCBpbnRlbGVjdHVhbCB5IHF1ZSBlbCByZXBvc2l0b3JpbyBpbnN0aXR1dGljaW9uYWwgZGUgbGEgVW5pdmVyc2lkYWQgQ29tcGx1dGVuc2UgZGUgTWFkcmlkLCBubyBhc3VtZSByZXNwb25zYWJpbGlkYWQgYWxndW5hIGVuIGNhc28gZGUgcXVlIHNlIHByb2R1emNhIHVuYSB2aW9sYWNpw7NuIGRlIGRlcmVjaG9zIGRlIHByb3BpZWRhZCBhbCBkaXN0cmlidWlyIGVzdG9zIGZpY2hlcm9zIG8gbWV0YWRhdG9zLgrigKIgVHJhYmFqb3MgZGVwb3NpdGFkb3MgcG9yIG90cm9zIHF1ZSBubyBzZWFuIHN1IGF1dG9yOiBkZWNsYXJvIHF1ZSBsb3MgbWV0YWRhdG9zIGJpYmxpb2dyw6FmaWNvcyBxdWUgZXN0b3kgYXJjaGl2YW5kbyBlbiBlbCByZXBvc2l0b3JpbyBpbnN0aXR1Y2lvbmFsIGRlIGxhIFVuaXZlcnNpZGFkIENvbXBsdXRlbnNlIGRlIE1hZHJpZCwgc29uIGRlIGRvbWluaW8gcMO6YmxpY28uIFNpIG5vIGZ1ZXNlIGVsIGNhc28sIGFjZXB0byBwbGVuYSByZXNwb25zYWJpbGlkYWQgcG9yIGN1YWxxdWllciBpbmZyYWNjacOzbiBkZSBkZXJlY2hvcyBkZSBhdXRvciBxdWUgY29ubGxldmUgbGEgZGlzdHJpYnVjacOzbiBkZSBlc3RvcyBmaWNoZXJvcyBvIG1ldGFkYXRvcy4K4oCiIFBvciB0YW50bywgdG9kb3MgbG9zIG1ldGFkYXRvcyBwdWJsaWNhZG9zIGVuIGVsIHJlcG9zaXRvcmlvIGluc3RpdHVjaW9uYWwgZGUgbGEgVW5pdmVyc2lkYWQgQ29tcGx1dGVuc2UgZGUgTWFkcmlkLCBlc3TDoW4gZGlzcG9uaWJsZXMgY29uIGxhIGxpY2VuY2lhIENyZWF0aXZlIENvbW1vbnMgQ0MwIDEuMCBVbml2ZXJzYWwgT2ZyZWNpbWllbnRvIGFsIERvbWluaW8gUMO6YmxpY28gKENDMCAxLjApLiBFc3RhIGxpY2VuY2lhIHBlcm1pdGUgY29waWFyLCBtb2RpZmljYXIsIGRpc3RyaWJ1aXIsIHJldXRpbGl6YXIgeSBjb21wYXJ0aXIgcMO6YmxpY2EgeSBsaWJyZW1lbnRlIGxvcyBtZXRhZGF0b3MgKGluY2x1eWVuZG8gc3UgdXNvIGNvbWVyY2lhbCksIHNpbiB0ZW5lciBxdWUgcGVkaXIgcGVybWlzby4gU2kgdGVyY2VyYXMgcGFydGVzIHVzYW4gZXN0b3MgbWV0YWRhdG9zLCBkZWJlbiBoYWNlciB1biByZWNvbm9jaW1pZW50byB5IHVuYSBhdHJpYnVjacOzbiBhIGxhcyBmdWVudGVzIGRlIGxvcyBtaXNtb3MuIExhIHJldXRpbGl6YWNpw7NuIGRlIGVzdG9zIG1ldGFkYXRvcyB0YW1iacOpbiBlcyBsaWJyZSwgdGFsIHkgY29tbyBlc3RhYmxlY2UgbGEgbGljZW5jaWEgZGUgdXNvIENDMC4K URL https://docta.ucm.es/bitstreams/b536c3e6-5141-44ed-9c95-b5ba1c767b00/download File MD5 1e4ff358dd3c3a4c62f31513c95b0097 8059518 application/pdf Hybrid-Cache_Computing_Servers.pdf URL https://docta.ucm.es/bitstreams/09327cf8-df5a-4d5c-b451-f103dbb787f3/download File MD5 f93eec70a03200cf4d87e5413c27c301 80024 text/plain Zapater_2022_reinforcement-learning-based.pdf.txt