MOVの機械学習システムを支えるMLOps実践

•

0 j'aime•214 vues

Takashi Suzuki

2020/03に行われたDeNA TechConの登壇資料

Ingénierie

DeNA TechCon 2020
#denatechcon
々
Map data ©2019 Google

DeNA TechCon 2020
#denatechcon
々
AI Platform TrainingBigQuery

DeNA TechCon 2020
#denatechcon
々
々
々
々々々

DeNA TechCon 2020
#denatechcon
def task_fail_slack_alert(context):
slack_msg = “Task Failed.”
failed_alert = SlackWebhookOperator(
task_id="xxx",
http_conn_id="slack",
webhook_token=Variable.get("webhook.token"),
message=slack_msg,
username="airflow",
)
return failed_alert.execute(context=context)

DeNA TechCon 2020
#denatechcon
<!subteam^ABCDEF123|@user_group>

DeNA TechCon 2020
#denatechcon
※
@pytest.fixture(scope="session", autouse=True)
def setup_teardown(request, test_config):
# setup
#
suffix = uuid.uuid4()
create_dataset(suffix)
yield
# teardown
#
delete_blobs(suffix)
delete_dataset(suffix)

DeNA TechCon 2020
#denatechcon
@pytest.mark.dependency(depends=[“test_preprocess_a”, “test_preprocess_b”])
def test_trainer(test_config):
trainer = Trainer(
test_config[“xxx”].format(test_config[“hash"]),
test_config[“yyy”].format(test_config[“hash"]),
)
trainer.train()
trainer.upload_model()

Contenu connexe

Similaire à MOVの機械学習システムを支えるMLOps実践

In the age of DevOps, “infrastructure as code” (IaC) is an emerging practice in which infrastructure is treated as versioned code. This practice reduces the effort required to create and manage resources and improves knowledge sharing between team members. Our presenter, CTO Andre Agostinho, will discuss IaC and how the automation of cloud services can meet the growing demands of businesses for development and scaling. This session will introduce IaC to participants and will cover the benefits and challenges of the practice. The presenter will also review the tools required and guide the audience through choosing the right IaC for every potential case. Lastly, participants will see a demonstration combining IaC with multi-cloud services and using continuous integration (CI) and continuous deployment (CD).

Scaling multi cloud with infrastructure as code

André Agostinho

Fairfax Sydney #mojo #mojocon Feb 2017 Meetup - 360 video production basics

Jamie Andrei

Behaviour-Driven Development (BDD) is a game changer for the whole team! More than just a testing technique, BDD is both a collaboration and a verification tool, and a vital step on the road to Continuous Delivery. In this session, you will learn what BDD is about, its benefits, and how it affects development teams and processes. But you will also see BDD techniques applied to a real project using tools like JBehave, Cucumber, Selenium 2, Thucydides and more! - Learn how BDD helps teams focus on discovering and delivering the features that really matter! - Learn what it takes to write more relevant and more maintainable automated acceptance tests - Discover how a well-designed set of automated acceptance criteria can also be a powerful documentation and reporting tool. - See where BDD fits into a Continuous Delivery pipeline. - And learn how product owners use BDD and Thucydides to drive, coordinate and document releases. Learn how much more there is to BDD than just “Given..When..Then”!

BDD in Action - Devoxx 2014

John Ferguson Smart Limited

Tekton showcase - CDF Summit Kubecon Barcelona 2019

Christie Wilson

The 4th annual Python Web Conf (#PWC2022) was a virtual event designed to promote best practices for hard web production problems. It featured international experts presenting on topics such as Django, CI/CD, Containers, Serverless, REST APIs, web security, microservices and more! The highly engaging format featured 90 speakers, 6 tracks (including 80 talks and 4 tutorials) and took place virtually from March 21-25, 2022 on LoudSwarm by Six Feet Up. More information about the 2022 conference can be found at: 2022.pythonwebconf.com.

Python Web Conf 2022 Event Statistics

Six Feet Up default

Sony Vegas Pro 13.0 Instruction Manual and Guide - English

Rahul Jose

Telegraf is an agent for collecting, processing, aggregating, and writing metrics. With over 200 plugins, Telegraf can fetch metrics from a variety of sources, allowing you to build aggregations and write those metrics to InfluxDB, Prometheus, Kafka, and more. In this talk, we will take a look at some of the lesser known, but awesome, plugins that are often overlooked; as well as how to use Telegraf for monitoring of Cloud Native systems.

OSMC 2019 | The Telegraf Toolbelt: It Can Do That, Really? by David McKay

NETWAYS

Web Real-Time Communications (WebRTC) has been rapidly growing for the past 3 years gaining adoption from many corporations and industries around the world. Surpassing the one billion mark for compatible devices, WebRTC has sparked some heated debate around whether or not it’s a truly disruptive technology. In this presentation we will discuss the possibilities and the current applications built and running full WebRTC services, despite the lack of support in Internet Explorer and Safari and the codecs wars.

WebRTC: The Big Debate, Shut Up and Build Something

Digium

Astricon 2014 - WebRTC - The Big Debate, I Say Shut Up and Build Something - ...

Blacc Spot Media, Inc.

Flutter Festival London 2022 - End to end IoT with Dart and Flutter

Chris Swan

Dektec

TELE-audiovision eng

Back-2-Basics: Code Contracts

David McCarter

WaveEngine Dotnet 2018

Javier Cantón Ferrero

Lightscribe

Nathan Wade

Graphics of the Americas 2017 Attendee Brochure

Depo Consulting

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2UgQ3BU. Christie Wilson describes what to expect from CI/CD in 2019, and how Tekton is helping bring that to as many tools as possible, such as Jenkins X and Prow. Wilson talks about Tekton itself and performs a live demo that shows how cloud native CI/CD can help debug, surface and fix mistakes faster. Filmed at qconsf.com. Christie Wilson is a software engineer at Google, currently leading the Tekton project. Over the past decade, she has worked in the mobile, financial and video game industries. Prior to working at Google she led a team of software developers to build load testing tools for AAA video game titles, and founded the Vancouver chapter of PyLadies.

Shifting Left with Cloud Native CI/CD

C4Media

FIWARE Global Summit - Real-time Media Stream Processing using Kurento

FIWARE

When we design together - Sabrina Mach, Ammara Gafoor and James Emmott From three distinct perspectives, this talk will contend that design is an activity undertaken by everyone in a software development team. It occurs throughout the process of delivery — not only at the beginning or the end — and it is a powerful instrument for learning about and adapting to the problems our work seeks to solve, which is a shared responsibility. Making the best use of our multidisciplinary expertise in the activity of design requires forms of collaboration that are too often disrupted by the role-based silos that keep us separated and weaken the valuable contribution our diverse approaches could make to our collective efforts. If you care about accelerating time to market, improving customer experience, or building happy and productive teams, you will want to know why and how it matters that we believe ‘design is in everything that we do’.

When we design together

Thoughtworks

Doodle maker review special offer - doodle maker features

rajugond2

WebRTC Reborn SignalConf 2016

Dan Jenkins

Similaire à MOVの機械学習システムを支えるMLOps実践 (20)

Scaling multi cloud with infrastructure as code

Fairfax Sydney #mojo #mojocon Feb 2017 Meetup - 360 video production basics

BDD in Action - Devoxx 2014

Tekton showcase - CDF Summit Kubecon Barcelona 2019

Python Web Conf 2022 Event Statistics

Sony Vegas Pro 13.0 Instruction Manual and Guide - English

OSMC 2019 | The Telegraf Toolbelt: It Can Do That, Really? by David McKay

WebRTC: The Big Debate, Shut Up and Build Something

Astricon 2014 - WebRTC - The Big Debate, I Say Shut Up and Build Something - ...

Flutter Festival London 2022 - End to end IoT with Dart and Flutter

Dektec

Back-2-Basics: Code Contracts

WaveEngine Dotnet 2018

Lightscribe

Graphics of the Americas 2017 Attendee Brochure

Shifting Left with Cloud Native CI/CD

FIWARE Global Summit - Real-time Media Stream Processing using Kurento

When we design together

Doodle maker review special offer - doodle maker features

WebRTC Reborn SignalConf 2016

Plus de Takashi Suzuki

到着予想時間サービスの特徴量のニアリアルタイム化

Takashi Suzuki

AI予約サービスのMLOps事例紹介

Takashi Suzuki

Kubernetes超入門

Takashi Suzuki

MLプロジェクトのリリースフローを考える

Takashi Suzuki

Optuna on Kubeflow Pipeline 分散ハイパラチューニング

Takashi Suzuki

GOの機械学習システムを支えるMLOps事例紹介

Takashi Suzuki

タクシー×AIを支えるKubernetesとAIデータパイプラインの信頼性の取り組みについて

Takashi Suzuki

MOV お客さま探索ナビの GCP ML開発フローについて

Takashi Suzuki

[Gree] DataEngConf NYC’18 セッションサマリー #1

Takashi Suzuki

[Gree] Dialogflowを利用したチャットボット導入事例

Takashi Suzuki

[Gree] グリーのソーシャルゲームにおける機械学習活用事例

Takashi Suzuki

Plus de Takashi Suzuki (11)

到着予想時間サービスの特徴量のニアリアルタイム化

AI予約サービスのMLOps事例紹介

Kubernetes超入門

MLプロジェクトのリリースフローを考える

Optuna on Kubeflow Pipeline 分散ハイパラチューニング

GOの機械学習システムを支えるMLOps事例紹介

タクシー×AIを支えるKubernetesとAIデータパイプラインの信頼性の取り組みについて

MOV お客さま探索ナビの GCP ML開発フローについて

[Gree] DataEngConf NYC’18 セッションサマリー #1

[Gree] Dialogflowを利用したチャットボット導入事例

[Gree] グリーのソーシャルゲームにおける機械学習活用事例

Dernier

5G and 6G refer to generations of mobile network technology, each representin...

archanaece3

Autodesk Construction Cloud (Autodesk Build).pptx

Mustafa Ahmed

Computer Graphics Introduction To Curves

ChandrakantDivate1

Ground Improvement Technique: Earth Reinforcement

Dr. Deepak Mudgal

Presentation on Slab, Beam, Column, and Foundation/Footing

Er. Suman Jyoti

Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf

JNTUA

Worksharing and 3D Modeling with Revit.pptx

Mustafa Ahmed

UNIT 4 PTRP final Convergence in probability.pptx

kalpana413121

Introduction to Artificial Intelligence ( AI)

ChandrakantDivate1

Theory of Time 2024 (Universal Theory for Everything)

Ramkumar k

Dr Mrs A A Miraje C Programming PPT.pptx

ProfAAMiraje

Artificial intelligence presentation2-171219131633.pdf

Kira Dess

We investigated the applicability and efficiency of the MLMC approach to the Henry-like problem with uncertain porosity, permeability and recharge. These uncertain parameters were modelled by random fields with three independent random variables. Permeability is a function of porosity. Both functions are time-dependent, have multi-scale behaviour and are defined for two layers. The numerical solution for each random realisation was obtained using the well-known ug4 parallel multigrid solver. The number of random samples required at each level was estimated by calculating the decay of the variances and the computational cost for each level. The MLMC method was used to compute the expected value and variance of several QoIs, such as the solution at a few preselected points $(t,\bx)$, the solution integrated over a small subdomain, and the time evolution of the freshwater integral. We have found that some QoIs require only 2-3 mesh levels and samples from finer meshes would not significantly improve the result. Other QoIs require more grid levels.

litvinenko_Henry_Intrusion_Hong-Kong_2024.pdf

Alexander Litvinenko

Path loss model, OKUMURA Model, Hata Model

DrAjayKumarYadav4

NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024

EMMANUELLEFRANCEHELI

Scouring of cotton and wool fabric with effective scouring method

vimal412355

Basics of Relay for Engineering Students

kannan348865

Max. shear stress theory-Maximum Shear Stress Theory Maximum Distortional ...

ronahami

8th International Conference on Soft Computing, Mathematics and Control (SMC 2024) will provide an excellent international forum for sharing knowledge and results in theory, methodology and applications impacts and challenges of Soft Computing, Mathematics and Control. The conference documents practical and theoretical results which make a fundamental contribution for the development of Soft Computing, Mathematics and Control. The aim of the conference is to provide a platform to the researchers and practitioners from both academia as well as industry to meet and share cutting-edge development in the field.

8th International Conference on Soft Computing, Mathematics and Control (SMC ...

josephjonse

Databricks Generative AI Fundamentals .pdf

VinayVadlagattu

Dernier (20)

5G and 6G refer to generations of mobile network technology, each representin...

Autodesk Construction Cloud (Autodesk Build).pptx

Computer Graphics Introduction To Curves

Ground Improvement Technique: Earth Reinforcement

Presentation on Slab, Beam, Column, and Foundation/Footing

Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf

Worksharing and 3D Modeling with Revit.pptx

UNIT 4 PTRP final Convergence in probability.pptx

Introduction to Artificial Intelligence ( AI)

Theory of Time 2024 (Universal Theory for Everything)

Dr Mrs A A Miraje C Programming PPT.pptx

Artificial intelligence presentation2-171219131633.pdf

litvinenko_Henry_Intrusion_Hong-Kong_2024.pdf

Path loss model, OKUMURA Model, Hata Model

NEWLETTER FRANCE HELICES/ SDS SURFACE DRIVES - MAY 2024

Scouring of cotton and wool fabric with effective scouring method

Basics of Relay for Engineering Students

Max. shear stress theory-Maximum Shear Stress Theory Maximum Distortional ...

8th International Conference on Soft Computing, Mathematics and Control (SMC ...

Databricks Generative AI Fundamentals .pdf

MOVの機械学習システムを支えるMLOps実践

1. !1

2. DeNA TechCon 2020 #denatechcon

3. !3

4. !4

5. DeNA TechCon 2020 #denatechcon ✔ ✔ ✔ 5

6. 6

7. 7

8. 8

9. 9 ● ● ● ●

10. 10 々

11. 11

12. 12

13. ● ● !13 ● ●

14. ● ● !14 ● / ● CI/CD ● ● ● ● 々

15. DeNA TechCon 2020 #denatechcon

16. !16 BigQuery 々

17. !17 BigQuery

18. DeNA TechCon 2020 #denatechcon

19. DeNA TechCon 2020 #denatechcon ※

20. DeNA TechCon 2020 #denatechcon ※

21. DeNA TechCon 2020 #denatechcon

22. DeNA TechCon 2020 #denatechcon ※ ？

23. DeNA TechCon 2020 #denatechcon ？

24. DeNA TechCon 2020 #denatechcon 々々 → 々

25. DeNA TechCon 2020 #denatechcon ※

26. !26 ※

27. !27 - ⇒ - - -

28. DeNA TechCon 2020 #denatechcon

29. !29 CI/CD   々

30. !30 CI/CD   々々

31. DeNA TechCon 2020 #denatechcon 々

32. DeNA TechCon 2020 #denatechcon 々 ※

33. !33

34. 々 - - - !34

35. DeNA TechCon 2020 #denatechcon

36. ● ※ ⇒   ※ ●   → ※ !36

37. DeNA TechCon 2020 #denatechcon

38. DeNA TechCon 2020 #denatechcon

39. DeNA TechCon 2020 #denatechcon 々※

40. DeNA TechCon 2020 #denatechcon

42. DeNA TechCon 2020 #denatechcon 々

43. DeNA TechCon 2020 #denatechcon

44. DeNA TechCon 2020 #denatechcon 々 AI Platform TrainingBigQuery

45. DeNA TechCon 2020 #denatechcon 々

46. DeNA TechCon 2020 #denatechcon

47. DeNA TechCon 2020 #denatechcon

48. DeNA TechCon 2020 #denatechcon

49. DeNA TechCon 2020 #denatechcon 々々々々々々

50. DeNA TechCon 2020 #denatechcon 々々々々々々？

51. DeNA TechCon 2020 #denatechcon 々々々々々々

52. DeNA TechCon 2020 #denatechcon ※ 々々

53. DeNA TechCon 2020 #denatechcon ※ 々

54. DeNA TechCon 2020 #denatechcon def task_fail_slack_alert(context): slack_msg = “Task Failed.” failed_alert = SlackWebhookOperator( task_id="xxx", http_conn_id="slack", webhook_token=Variable.get("webhook.token"), message=slack_msg, username="airflow", ) return failed_alert.execute(context=context)

55. DeNA TechCon 2020 #denatechcon <!subteam^ABCDEF123|@user_group>

56. DeNA TechCon 2020 #denatechcon 々々々々々々

57. DeNA TechCon 2020 #denatechcon 々

58. DeNA TechCon 2020 #denatechcon ※

59. DeNA TechCon 2020 #denatechcon

60. DeNA TechCon 2020 #denatechcon

61. DeNA TechCon 2020 #denatechcon

62. DeNA TechCon 2020 #denatechcon 々々

63. DeNA TechCon 2020 #denatechcon 々

64. DeNA TechCon 2020 #denatechcon

65. DeNA TechCon 2020 #denatechcon 々

66. DeNA TechCon 2020 #denatechcon ※

67. DeNA TechCon 2020 #denatechcon → 々

68. DeNA TechCon 2020 #denatechcon 々

69. DeNA TechCon 2020 #denatechcon ※ @pytest.fixture(scope="session", autouse=True) def setup_teardown(request, test_config): # setup # suffix = uuid.uuid4() create_dataset(suffix) yield # teardown # delete_blobs(suffix) delete_dataset(suffix)

70. DeNA TechCon 2020 #denatechcon @pytest.mark.dependency(depends=[“test_preprocess_a”, “test_preprocess_b”]) def test_trainer(test_config): trainer = Trainer( test_config[“xxx”].format(test_config[“hash"]), test_config[“yyy”].format(test_config[“hash"]), ) trainer.train() trainer.upload_model()

71. DeNA TechCon 2020 #denatechcon →

72. DeNA TechCon 2020 #denatechcon 々 →

73. ● ＝ ● ＝ ● !73

74. DeNA TechCon 2020

MOVの機械学習システムを支えるMLOps実践

Recommandé

Recommandé

Contenu connexe

Similaire à MOVの機械学習システムを支えるMLOps実践

Similaire à MOVの機械学習システムを支えるMLOps実践 (20)

Plus de Takashi Suzuki

Plus de Takashi Suzuki (11)

Dernier

Dernier (20)

MOVの機械学習システムを支えるMLOps実践