ICSE 24, FTW: On the Impact of Hitting System Resource Limits on Test Flakiness.pptx

•Télécharger en tant que PPTX, PDF•

0 j'aime•3 vues

Slides used by Alexander Perathoner in his presentation of the paper "On the Impact of Hitting System Resource Limits on Test Flakiness". Presentation took place on 14th April 2024 at the International Flaky Test Workshop, co-located at the International Conference on Software Engineering (ICSE 24) in Lisbon. The paper was written by F. Leinen, A. Perathoner and Prof. Dr. A. Pretschner.

Présentations et discours publics

On the Impact of Hitting System
Resource Limits on Test Flakiness
FTW 2024
April 14, 2024
Fabian Leinen Alexander Perathoner Alexander
Pretschner Technical University Munich

Motivation
Intuition: “PC under load provokes lags and flakiness”
Study Gap: resources are not stressed, just measured
Hypothesis: “Reaching CPU limits promotes timing-related
flakiness”
2

Approach: UI Tests & macOS
Weaker Resources
More expensive
3

Does Hitting Natural Resource Limits Impact Flakiness?
We measure ratio of:
- time at resource limit
- total runtime
in executions of flaky / non-flaky tests
5

Does Hitting Natural Resource Limits Impact Flakiness?
We measure ratio of:
- time at resource limit
- total runtime
in passing / failing executions of flaky tests
6

Gathering Data
select:repo
(type:file content:XCUIApplication lang:Swift)
(repo:has.tag(macos) OR
repo:has.tag(osx) OR
repo:has.topic(macOS))
236 repositories
7

236 repositories
Manual filtering
20 projects
Gathering Data
6.5k
Sparkle
8.1k
Maccy
8

tum-i4/workflow-telemetry-action
measuring:
CPU, RAM, network, disk
GitHub
Workflow
Gathering Data in the GitHub CI
9

Does Hitting Natural Resource Limits Impact Flakiness?
13

Does Hitting Natural Resource Limits Impact Flakiness?
14

Result Discussion
Alternative explanations for RQ1:
- Flakiness linked to complexity of tests (size) rather
than resource limits?
No quantitative explanations for RQ2
15

What to Take Away From This
We analyzed the impact of resource limits:
1. on tests being flaky → they do spend more time at
limits
2. on flaky failures → they do not
16
Upgrade the server to boost stability
But don't expect it to magically ace tests

Call for Action
● More projects!
● More resources!
● Less overhead!
Remember the GitHub Action!
17

Contenu connexe

Similaire à ICSE 24, FTW: On the Impact of Hitting System Resource Limits on Test Flakiness.pptx

Advances in genomics and data analytics create new opportunities for cancer research and personalized medical treatment via large-scale federation of genomic, clinical, imaging and other data from many thousands of patients across institutions around the world. Despite these opportunities and promising early results, cancer research is often stymied by information technology barriers. One major barrier is a lack of tools for the reliable, secure, rapid, and easy transfer, sharing, and management of large collections of human data. In the absence of such tools, security and performance concerns often prevent sharing altogether or force researchers to resort to slow and error prone shipping of physical media. If data are received, timely analysis is further impeded by the difficulties inherent in verifying data integrity and managing who can access data and for what purpose. I will discuss how the mature Globus data management platform addresses these obstacles to discovery and explain how its intuitive, web-based interfaces enable use by researchers without specialized IT knowledge. I also describe how Globus technologies can be extended to meet the security requirements of human data so as to enable use in data-intensive cancer research.

Streamlined data sharing and analysis to accelerate cancer research

Ian Foster

The proliferation of mobile devices in the form of smartphones and tablet devices has put into our hands computational power and capability previously limited to desktop until recently. Couple this with the connectivity of these devices to the internet and the trend of increased capability and accessibility in smaller devices continues. This presentation will provide an overview of our efforts to provide access on mobile devices to a large chemistry database, ChemSpider, containing over 25 million unique chemical compounds and associated data including patents, publications, properties and analytical data. We will also discuss how, by providing programming interfaces and Open Data, it has been possible to produce a Spectral Game (www.spectralgame.com) for scientists and students to practice their spectral interpretation skills. We will discuss some of the technology hurdles associated with delivering such capabilities to the various mobile platforms and how modern technologies can significantly enhance the user experience.

Chemistry in the hand: The delivery of structure databases and spectroscopy g...

US Environmental Protection Agency (EPA), Center for Computational Toxicology and Exposure

Among Viruses, Trojans, and Backdoors:Fighting Malware in 2022

Marcus Botacin

On the Malware Detection Problem: Challenges & Novel Approaches

Marcus Botacin

"Data Provenance: Principles and Why it matters for BioMedical Applications"

Pinar Alper

Break -> inspect -> fix is the Ops workflow for infrastructure stacks of the past. Distributed infrastructure and applications claim to be the new generation, but why is it so much more painful to maintain and troubleshoot them? Much of the pain comes from outdated operational models relying on reactive or, worse yet, manual monitoring and Ops. This talk lays out a proactive Ops model for container infrastructure. By focusing on event monitoring, infrastructure state monitoring, trend analysis, and distributed log collection, a proactive Ops model delivers observability for distributed apps that was not possible before. Using real-world examples from Swarm and Kubernetes, we'll demonstrate the tools used and how we relieve Ops pain in container orchestration.

Proactive ops for container orchestration environments

Docker, Inc.

Belak_ICME_June02015

Jim Belak

OpenPOWER Academia and Research team's webinar - Presentations from Oak Ridg...

Ganesan Narayanasamy

[A talk presented at Oak Ridge National Laboratory on October 15, 2015] We have made much progress over the past decade toward harnessing the collective power of IT resources distributed across the globe. In big-science projects in high-energy physics, astronomy, and climate, thousands work daily within virtual computing systems with global scope. But we now face a far greater challenge: Exploding data volumes and powerful simulation tools mean that many more--ultimately most?--researchers will soon require capabilities not so different from those used by such big-science teams. How are we to meet these needs? Must every lab be filled with computers and every researcher become an IT specialist? Perhaps the solution is rather to move research IT out of the lab entirely: to develop suites of science services to which researchers can dispatch mundane but time-consuming tasks, and thus to achieve economies of scale and reduce cognitive load. I explore the past, current, and potential future of large-scale outsourcing and automation for science, and suggest opportunities and challenges for today’s researchers. I use examples from Globus and other projects to demonstrate what can be achieved.

Accelerating Discovery via Science Services

Ian Foster

Domains such as drug discovery, data science, and policy studies increasing rely on the combination of complex analysis pipelines with integrated data sources to come to conclusions. A key question then arises is what are these conclusions based upon? Thus, there is a tension between integrating data for analysis and understanding where that data comes from (its provenance). In this talk, I describe recent work that is attempting to facilitate transparency by combining provenance tracked within databases with the data integration and analytics pipelines that feed them. I discuss this with respect to use cases from public policy as well as drug discovery. Given at: http://ccct.uva.nl/content/ccct-seminar-21-february-2014

Transparency in the Data Supply Chain

Paul Groth

Accelerating Data-driven Discovery in Energy Science

Ian Foster

Performance Analysis of Idle Programs

greenwop

PhD public defense: A Measurement Framework for Analyzing Technical Lag in ...

Ahmed Zerouali

Windows Live Forensics 101

Arpan Raval

Containers in Science: neuroimaging use cases

Krzysztof Gorgolewski

Current Topics paper A4 submission 4.30.2015 Master Copy

Tommie Walls

Practical Chaos Engineering

SIGHUP

Virtual Gov Day - IT Operations Breakout - Jennifer Green, R&D Scientist, Los...

Splunk

Stream based data / event / message processing becomes preferred way of achieving interoperability and real-time communication in distributed SOA / microservice / database architectures. Beside lambdas, Java 8 introduced two new APIs explicitly dealing with stream data processing: - Stream - which is PULL-based and easily parallelizable; - CompletableFuture / CompletionStage - which allow composition of PUSH-based, non-blocking, asynchronous data processing pipelines. Java 9 will provide further support for stream-based data-processing by extending the CompletableFuture with additional functionality – support for delays and timeouts, better support for subclassing, and new utility methods. More, Java 9 provides new java.util.concurrent.Flow API implementing Reactive Streams specification that enables reactive programming and interoperability with libraries like Reactor, RxJava, RabbitMQ, Vert.x, Ratpack, and Akka. The presentation will discuss the novelties in Java 8 and Java 9 supporting stream data processing, describing the APIs, models and practical details of asynchronous pipeline implementation, error handling, multithreaded execution, asyncronous REST service implementation, interoperability with existing libraries. There are provided demo examples (code on GitHub) using Completable Future and Flow with: - JAX-RS 2.1 AsyncResponse, and more importantly unit-testing the async REST service method implementations; - CDI 2.0 asynchronous observers (fireAsync / @ObservesAsync);

Stream Processing with CompletableFuture and Flow in Java 9

Trayan Iliev

Cloud Resilience with Open Stack

Jorge Cardoso

Similaire à ICSE 24, FTW: On the Impact of Hitting System Resource Limits on Test Flakiness.pptx (20)

Streamlined data sharing and analysis to accelerate cancer research

Chemistry in the hand: The delivery of structure databases and spectroscopy g...

Among Viruses, Trojans, and Backdoors:Fighting Malware in 2022

On the Malware Detection Problem: Challenges & Novel Approaches

"Data Provenance: Principles and Why it matters for BioMedical Applications"

Proactive ops for container orchestration environments

Belak_ICME_June02015

OpenPOWER Academia and Research team's webinar - Presentations from Oak Ridg...

Accelerating Discovery via Science Services

Transparency in the Data Supply Chain

Accelerating Data-driven Discovery in Energy Science

Performance Analysis of Idle Programs

PhD public defense: A Measurement Framework for Analyzing Technical Lag in ...

Windows Live Forensics 101

Containers in Science: neuroimaging use cases

Current Topics paper A4 submission 4.30.2015 Master Copy

Practical Chaos Engineering

Virtual Gov Day - IT Operations Breakout - Jennifer Green, R&D Scientist, Los...

Stream Processing with CompletableFuture and Flow in Java 9

Cloud Resilience with Open Stack

Dernier

Report Writing Webinar Training

KylaCullinane

Chiulli_Aurora_Oman_Raffaele_Beowulf.pptx

raffaeleoman

Mohammad_Alnahdi_Oral_Presentation_Assignment.pptx

mohammadalnahdi22

Causes of poverty in France presentation.pptx

CamilleBoulbin1

Re-membering the Bard: Revisiting The Compleat Wrks of Wllm Shkspr (Abridged)...

Hasting Chen

lONG QUESTION ANSWER PAKISTAN STUDIES10.

lodhisaajjda

💚❤️Call girl in Chandigarh ☎️8868886958☎️ Call Girl service in Chandigarh☎️ Chandigarh Call Girls Service ☎️ Call Girls In Chandigarh Call Girl Service In Chandigarh 💯Call Ruhi 🔝8868886958🔝Chandigarh Call Girl WhatsApp Chat: ☎️ +91-8868886958 Call Girls In Chandigarh offer you everything from intimate moments to wild nights - be it intimate or wild. Their girls are always prepared to meet all your needs and desires so you're assured a memorable experience with them. Beautiful Babes are sure to turn heads wherever they go, captivating men with their seductive personalities and captivating looks - not to mention their sensually alluring bodies that are sure to satisfy all your naughty fantasies, from blowjobs to anal massages; not forgetting BDSM and orgies as well. Show them some appreciation when the time is right. Call Girl Chandigarh that their clients can be very demanding. To maintain their reputation and gain more clients, these sexy call girls always work tirelessly to provide exceptional service that ensures each customer's happiness - so much so that many men seek them out to have an unforgettable experience with Passionate about providing erotic pleasure for their clients - fulfilling all sexy fantasies while offering role play services such as BDSM. ★OUR BEST SERVICES: - FOR BOOKING ★ A-Level (5-star escort) ★ Strip-tease ★ BBBJ (Bareback Blowjob) ★ Spending time in my rooms ★ BJ (Blowjob Without a Condom) ★. Extra ball (Have ride many times) ☛ ☛ ☛ ✔✔ secure✔✔ 100% safe WHATSAPP CALL ME💥✨✨⭐⭐⚡💦💦💨🔥💫💫 Their services range from erotic encounters and movie dates to in-call and out-call services, making this option available 24/7. Available for services including NSA, threesomes and foursomes sessions as well as massage services and casual foreplay - they even provide real girlfriend experience. Find one online or visit local women's clubs, hotels or restaurants. Hiring a Chandigarh call girl for an evening or day out can be the perfect addition to your social gathering or office event, or you could arrange for her to visit your home or hotel room. Not only are these women gorgeous, they're intelligent as well, with great senses of humor - sure to please and leave you wanting more. If you want a cheap escort in Chandigarh, look for someone who is either a student or works part time so that she will always be available when you need her. In addition to this, look for housewives as this ensures they have stable lives that can keep you satisfied for extended periods. These beautiful ladies will capture your attention at first sight with stunning eyes and full, seductive lips; not to mention a seductive personality that makes you want to spend more time with them; moreover they are discreet enough to meet up anywhere nearby! Hiring a call girl in Chandigarh can be done through either the internet or calling her directly, with services like massage or sex sessions offered to request from specific agencies within.

No Advance 8868886958 Chandigarh Call Girls , Indian Call Girls For Full Nigh...

Sheetaleventcompany

ICT role in 21st century education and it's challenges.pdf

Islamia university of Rahim Yar khan campus

Yesterday in Lagos, I had the honour of delivering a lecture titled “If this Giant Must Walk; Manifesto for a New Nigeria“ at the Inaugural Memorial Lecture of Prince Emeka Obasi, founder and publisher of Business Hallmark and the inspirational figure behind the Public Policy Research and Analysis Centre - promoters of the revered Zik Leadership and Governance Awards. My lecture focused on the challenges of nation-building in Nigeria and how we can approach them in a way that promotes progress and unity. I discussed the many sources of concern about our country’s future prospects, including violent conflicts, revisionist contestations of the Amalgamation Act of 1914, discontent with the Nigerian economy, and dysfunctions in the federal system. Central to my lecture was the call to address these challenges by crafting a new manifesto for Nigeria. This manifesto, I proposed, should champion integrity, compassion, character, competence, and commitment to national unity and progress within a framework of democratic governance and cultural diversity. I firmly believe that by doing so, we can guide Nigeria to stride forward with pride and purpose. I want to thank the Board and Management of the Public Policy Forum for their kind invitation to speak at this important event and for their commitment to promoting public policy research and analysis in Nigeria. My gratitude also to the late Prince Emeka Obasi, a true inspiration and a builder of bridges across divides, for his contributions to the country. My thoughts are with his family and loved ones.

If this Giant Must Walk: A Manifesto for a New Nigeria

Kayode Fayemi

My Presentation "In Your Hands" by Halle Bailey

hlharris

(Vivek)Call Us, 8448380779,Call girls in Delhi NCr – We Offer best in class call girls. escort Service At Affordable Price At low Rate with Space Night 8000 We Are One Of The Oldest Escort and Call girls Agencies in Delhi. You Will Find That Our Female Escorts Are Full Of Fun, Sexy And They Would Love Enjoy Your Company. We Have A Fantastic Selection Of Escort Ladies Available For In-Calls As Well As Out-Calls. Our Escorts Are Not Only Beautiful But All Have Great Personalities Making Them The Perfect Companion For Any Occasion. In-Call:- You Can Come At Our Place in Delhi Our place Which Is Very Clean Hygienic 100% safe Accommodation. Out-Call:- You have To Come Pick The Girl From My Place We Are Also Provide Door Step Services (Delhi Ncr, Noida, Gurgaon, Faridabad, Ghaziabad Note:- Pic Collectors Time Passers Bargainers Stay Away As We Respect The Value For Your Money Time And Expect The Same From You Hygienic:- Full Ac room And Clean Rooms Available In Hotel 24 * 7 Hourly In Delhi NCR More Details, With WhatsApp Number, +91-8448380779

Busty Desi⚡Call Girls in Sector 51 Noida Escorts >༒8448380779 Escort Service-...

Delhi Call girls

VVIP Call Girls Nalasopara : 9892124323, Call Girls in Nalasopara Services

Pooja Nehwal

• For a full set of 390+ questions. Go to https://skillcertpro.com/product/aws-data-engineer-associate-dea-c01-exam-questions/ • SkillCertPro offers detailed explanations to each question which helps to understand the concepts better. • It is recommended to score above 85% in SkillCertPro exams before attempting a real exam. • SkillCertPro updates exam questions every 2 weeks. • You will get life time access and life time free updates • SkillCertPro assures 100% pass guarantee in first attempt.

AWS Data Engineer Associate (DEA-C01) Exam Dumps 2024.pdf

SkillCertProExams

BDSM⚡Call Girls in Sector 93 Noida Escorts >༒8448380779 Escort Service

Delhi Call girls

I had the honour of delivering the Keynote Address at the 34th Annual Conference of the Nigerian Political Science Association (NPSA) in Lokoja. The conference theme, "Governance and Nation-Building in Nigeria: Some Reflections on Options for Policy," is relevant and timely. The challenges of nation-building are manifesting worldwide, including our continent and country. Violent conflicts and subterranean discord and discontent are prevalent, and the revival of irredentist ethno-regional and religious identities further complicates the situation. I believe that the conference was a great opportunity to discuss these challenges and reflect on practical policy options. I salute the President of the NPSA, Professor Hassan Saliu, for his leadership and the Executive Committee's effort to mobilize the membership despite a paucity of resources and the prevailing tough economic times. I am optimistic that the NPSA will continue to thrive, and I urge all of us to work towards the continued stability and unity of our country.

Governance and Nation-Building in Nigeria: Some Reflections on Options for Po...

Kayode Fayemi

The workplace ecosystem of the future 24.4.2024 Fabritius_share ii.pdf

Senaatti-kiinteistöt

Introduction to Prompt Engineering (Focusing on ChatGPT)

Chameera Dedduwage

Presentation on Engagement in Book Clubs

samaasim06

I had the honour of presenting my reflections on the autobiography of Professor Isaac Folorunso Adewole, titled "Uncommon Grace." As someone deeply invested in documentation and history, I found Professor Adewole's decision to narrate his journey from humble beginnings to occupying one of the highest offices in the land both inspiring and invaluable. In this eloquently written memoir, Professor Adewole provides a comprehensive account of his life, from his ancestral roots to his time as Minister of Health in Nigeria. The unique aspect of this autobiography is that he portrayed himself authentically without taking the help of third-party narratives, which is often seen in accounts of high-ranking officials. His upbringing was greatly influenced by his father's commitment to education. He became a prominent figure in advocating for the rights of the underprivileged through trade unionism. His story is one of unwavering determination, resilience, and faith. His experiences, including both successes and struggles, provide priceless lessons on leadership, perseverance, and the alignment of personal values with public service. While reading "Uncommon Grace," I was struck by the deep leadership lessons that are embedded within its chapters. Professor Adewole stresses the importance of inclusivity, servant leadership, and planning, which are all highly relevant in today's complex world. His commitment to accountability, as well as his primary responsibility as a researcher, serves as a guiding light for aspiring leaders across various disciplines. During his tenure as the Vice Chancellor of the University of Ibadan, he led with visionary leadership and transformative impact. His accomplishments have been meticulously documented in the book, which can serve as a blueprint for rejuvenating institutions and promoting academic excellence. In the latter part of his autobiography, Professor Adewole shares his experiences as a Minister, detailing the challenges he faced while serving the public with integrity and courage. His reflections on the complexities of public service, coupled with his commitment to the well-being of the nation, offer practical insights for policymakers and citizens alike. I have carefully read "Uncommon Grace" and it is more than just a memoir. It is a timeless book that is hard to put down once you start reading. While intellectuals may continue to debate whether uncommon grace was made possible by uncommon preparation or the other way around, I applaud Professor Adewole for sharing his ideas, knowledge, and experience with the public. I highly recommend this book to everyone.

Uncommon Grace The Autobiography of Isaac Folorunso

Kayode Fayemi

Dreaming Music Video Treatment _ Project & Portfolio III

NhPhngng3

Dernier (20)

Report Writing Webinar Training

Chiulli_Aurora_Oman_Raffaele_Beowulf.pptx

Mohammad_Alnahdi_Oral_Presentation_Assignment.pptx

Causes of poverty in France presentation.pptx

Re-membering the Bard: Revisiting The Compleat Wrks of Wllm Shkspr (Abridged)...

lONG QUESTION ANSWER PAKISTAN STUDIES10.

No Advance 8868886958 Chandigarh Call Girls , Indian Call Girls For Full Nigh...

ICT role in 21st century education and it's challenges.pdf

If this Giant Must Walk: A Manifesto for a New Nigeria

My Presentation "In Your Hands" by Halle Bailey

Busty Desi⚡Call Girls in Sector 51 Noida Escorts >༒8448380779 Escort Service-...

VVIP Call Girls Nalasopara : 9892124323, Call Girls in Nalasopara Services

AWS Data Engineer Associate (DEA-C01) Exam Dumps 2024.pdf

BDSM⚡Call Girls in Sector 93 Noida Escorts >༒8448380779 Escort Service

Governance and Nation-Building in Nigeria: Some Reflections on Options for Po...

The workplace ecosystem of the future 24.4.2024 Fabritius_share ii.pdf

Introduction to Prompt Engineering (Focusing on ChatGPT)

Presentation on Engagement in Book Clubs

Uncommon Grace The Autobiography of Isaac Folorunso

Dreaming Music Video Treatment _ Project & Portfolio III

ICSE 24, FTW: On the Impact of Hitting System Resource Limits on Test Flakiness.pptx

1. On the Impact of Hitting System Resource Limits on Test Flakiness FTW 2024 April 14, 2024 Fabian Leinen Alexander Perathoner Alexander Pretschner Technical University Munich

2. Motivation Intuition: “PC under load provokes lags and flakiness” Study Gap: resources are not stressed, just measured Hypothesis: “Reaching CPU limits promotes timing-related flakiness” 2

3. Approach: UI Tests & macOS Weaker Resources More expensive 3

4. Identifying Potentially Flaky Tests 4

5. Does Hitting Natural Resource Limits Impact Flakiness? We measure ratio of: - time at resource limit - total runtime in executions of flaky / non-flaky tests 5

6. Does Hitting Natural Resource Limits Impact Flakiness? We measure ratio of: - time at resource limit - total runtime in passing / failing executions of flaky tests 6

7. Gathering Data select:repo (type:file content:XCUIApplication lang:Swift) (repo:has.tag(macos) OR repo:has.tag(osx) OR repo:has.topic(macOS)) 236 repositories 7

8. 236 repositories Manual filtering 20 projects Gathering Data 6.5k Sparkle 8.1k Maccy 8

9. tum-i4/workflow-telemetry-action measuring: CPU, RAM, network, disk GitHub Workflow Gathering Data in the GitHub CI 9

10. 10 Gathering Data in the GitHub CI

11. 11 Gathering Data in the GitHub CI

12. How Does the Dataset Look? 12

13. Does Hitting Natural Resource Limits Impact Flakiness? 13

14. Does Hitting Natural Resource Limits Impact Flakiness? 14

15. Result Discussion Alternative explanations for RQ1: - Flakiness linked to complexity of tests (size) rather than resource limits? No quantitative explanations for RQ2 15

16. What to Take Away From This We analyzed the impact of resource limits: 1. on tests being flaky → they do spend more time at limits 2. on flaky failures → they do not 16 Upgrade the server to boost stability But don't expect it to magically ace tests

17. Call for Action ● More projects! ● More resources! ● Less overhead! Remember the GitHub Action! 17

18. Root Causes of Flaky Tests 18

ICSE 24, FTW: On the Impact of Hitting System Resource Limits on Test Flakiness.pptx

Recommandé

Recommandé

Contenu connexe

Similaire à ICSE 24, FTW: On the Impact of Hitting System Resource Limits on Test Flakiness.pptx

Similaire à ICSE 24, FTW: On the Impact of Hitting System Resource Limits on Test Flakiness.pptx (20)

Dernier

Dernier (20)

ICSE 24, FTW: On the Impact of Hitting System Resource Limits on Test Flakiness.pptx