Download as PDF, PPTX
Uploaded byNAVER D2
[212] large scale backend service develpment
The document discusses developing large scale backend services for a new game using Node.js, Docker, and AWS. It describes problems with scalability and performance, and solutions using Node.js clustering, reverse proxying, and CPU profiling to optimize services. The goals are to build services that are scalable, have high performance, and allow for fast iterations.
More Related Content
![[245] presto 내부구조 파헤치기](https://cdn.slidesharecdn.com/ss_thumbnails/245presto-150915054242-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[231] the simplicity of cluster apps with circuit](https://cdn.slidesharecdn.com/ss_thumbnails/213thesimplicityofclusterappswithcircuit-150915022150-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[233] level 2 network programming using packet ngin rtos](https://cdn.slidesharecdn.com/ss_thumbnails/233level2networkprogrammingusingpacketnginrtos-150915022357-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[221] docker orchestration](https://cdn.slidesharecdn.com/ss_thumbnails/212dockerorchestration-150915010646-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[261] 실시간 추천엔진 머신한대에 구겨넣기](https://cdn.slidesharecdn.com/ss_thumbnails/216-150915054828-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[252] 증분 처리 플랫폼 cana 개발기](https://cdn.slidesharecdn.com/ss_thumbnails/225cana-150915052201-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[244] 분산 환경에서 스트림과 배치 처리 통합 모델](https://cdn.slidesharecdn.com/ss_thumbnails/244-150915025618-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[241] Storm과 Elasticsearch를 활용한 로깅 플랫폼의 실시간 알람 시스템 구현](https://cdn.slidesharecdn.com/ss_thumbnails/83713-150915040003-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[224] 번역 모델 기반_질의_교정_시스템](https://cdn.slidesharecdn.com/ss_thumbnails/242-150915010843-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[222]대화 시스템 서비스 동향 및 개발 방법](https://cdn.slidesharecdn.com/ss_thumbnails/222-150915011307-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[211] 네이버 검색과 데이터마이닝](https://cdn.slidesharecdn.com/ss_thumbnails/211-150915001301-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[251] implementing deep learning using cu dnn](https://cdn.slidesharecdn.com/ss_thumbnails/215implementingdeeplearningusingcudnn-150915052020-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[213] ethereum](https://cdn.slidesharecdn.com/ss_thumbnails/231ethereum-150915024025-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[242] wifi를 이용한 실내 장소 인식하기](https://cdn.slidesharecdn.com/ss_thumbnails/224wifi-150915025827-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[262] netflix 빅데이터 플랫폼](https://cdn.slidesharecdn.com/ss_thumbnails/226netflix-150915054913-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[263] s2graph large-scale-graph-database-with-hbase-2](https://cdn.slidesharecdn.com/ss_thumbnails/236s2graph-large-scale-graph-database-with-hbase-2-150915055019-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[214] data science with apache zeppelin](https://cdn.slidesharecdn.com/ss_thumbnails/241datasciencewithapachezeppelin-150915001420-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[264] large scale deep-learning_on_spark](https://cdn.slidesharecdn.com/ss_thumbnails/246large-scaledeeplearningonspark-150915055051-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[243] turning data into value](https://cdn.slidesharecdn.com/ss_thumbnails/234turningdataintovalue-150915052705-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[234] 산업 현장을 위한 증강 현실 기기 daqri helmet 개발기](https://cdn.slidesharecdn.com/ss_thumbnails/243daqrihelmet-150915022636-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[232] 수퍼컴퓨팅과 데이터 어낼리틱스](https://cdn.slidesharecdn.com/ss_thumbnails/223-150915022242-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
![[253] apache ni fi](https://cdn.slidesharecdn.com/ss_thumbnails/235apachenifi-150915053924-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
![[223] h base consistent secondary indexing](https://cdn.slidesharecdn.com/ss_thumbnails/232hbaseconsistentsecondaryindexing-150915010743-lva1-app6891-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to [212] large scale backend service develpment
![[AWSKRUG 아키텍처 모임] 세일즈부스트 인프라스트럭처 사례 공유](https://cdn.slidesharecdn.com/ss_thumbnails/awskrug20180227-180228005405-thumbnail.jpg?width=640&height=640&fit=bounds)
![[225]NSML: 머신러닝 플랫폼 서비스하기 & 모델 튜닝 자동화하기](https://cdn.slidesharecdn.com/ss_thumbnails/225nsmlmachinelearningntuningautomize-181012023407-thumbnail.jpg?width=640&height=640&fit=bounds)
![[224]네이버 검색과 개인화](https://cdn.slidesharecdn.com/ss_thumbnails/224naversearchnpersonalizationfinal-181012022631-thumbnail.jpg?width=640&height=640&fit=bounds)
![[211] 인공지능이 인공지능 챗봇을 만든다](https://cdn.slidesharecdn.com/ss_thumbnails/211chatbot-181106094835-thumbnail.jpg?width=640&height=640&fit=bounds)
![[215] Druid로 쉽고 빠르게 데이터 분석하기](https://cdn.slidesharecdn.com/ss_thumbnails/215druid-181012071910-thumbnail.jpg?width=640&height=640&fit=bounds)
![[214] Ai Serving Platform: 하루 수 억 건의 인퍼런스를 처리하기 위한 고군분투기](https://cdn.slidesharecdn.com/ss_thumbnails/214aiservingplatforminference-181012022603-thumbnail.jpg?width=640&height=640&fit=bounds)
![[233] 대형 컨테이너 클러스터에서의 고가용성 Network Load Balancing: Maglev Hashing Scheduler i...](https://cdn.slidesharecdn.com/ss_thumbnails/233networkloadbalancing-181018151852-thumbnail.jpg?width=640&height=640&fit=bounds)
![Old version: [233]대형 컨테이너 클러스터에서의 고가용성 Network Load Balancing](https://cdn.slidesharecdn.com/ss_thumbnails/233largecontainerclusternetworkloadbalancing-181012024225-thumbnail.jpg?width=640&height=640&fit=bounds)
![[223]기계독해 QA: 검색인가, NLP인가?](https://cdn.slidesharecdn.com/ss_thumbnails/2232018-181012010149-thumbnail.jpg?width=640&height=640&fit=bounds)
![[232] TensorRT를 활용한 딥러닝 Inference 최적화](https://cdn.slidesharecdn.com/ss_thumbnails/232dlinferenceoptimizationusingtensorrt1-181012014455-thumbnail.jpg?width=640&height=640&fit=bounds)
![[242]컴퓨터 비전을 이용한 실내 지도 자동 업데이트 방법: 딥러닝을 통한 POI 변화 탐지](https://cdn.slidesharecdn.com/ss_thumbnails/242pcdpublic-181012011734-thumbnail.jpg?width=640&height=640&fit=bounds)
![[216]Search Reliability Engineering (부제: 지진에도 흔들리지 않는 네이버 검색시스템)](https://cdn.slidesharecdn.com/ss_thumbnails/216sresearchreliabilityengineering-181012022623-thumbnail.jpg?width=640&height=640&fit=bounds)
![[236] 스트림 저장소 최적화 이야기: 아파치 드루이드로부터 얻은 교훈](https://cdn.slidesharecdn.com/ss_thumbnails/236deview2018jihoonson-final-181012031726-thumbnail.jpg?width=640&height=640&fit=bounds)
![[243] Deep Learning to help student’s Deep Learning](https://cdn.slidesharecdn.com/ss_thumbnails/243deeplearningtohelpstudentsdeeplearning-181012024530-thumbnail.jpg?width=640&height=640&fit=bounds)
![[244]로봇이 현실 세계에 대해 학습하도록 만들기](https://cdn.slidesharecdn.com/ss_thumbnails/244deview2018tomisilanderrobotsrealworldfinal11oct2018-181012024720-thumbnail.jpg?width=640&height=640&fit=bounds)
![[234]Fast & Accurate Data Annotation Pipeline for AI applications](https://cdn.slidesharecdn.com/ss_thumbnails/234fastaccuratedataannotationpipelineforaiapplications1-181012024230-thumbnail.jpg?width=640&height=640&fit=bounds)
![[235]Wikipedia-scale Q&A](https://cdn.slidesharecdn.com/ss_thumbnails/235deview2018julienperezwikipediaqa12oct2018-181012030613-thumbnail.jpg?width=640&height=640&fit=bounds)
![[213] Fashion Visual Search](https://cdn.slidesharecdn.com/ss_thumbnails/213fashionvisualsearchreduced-181012022540-thumbnail.jpg?width=640&height=640&fit=bounds)
![[212]C3, 데이터 처리에서 서빙까지 가능한 하둡 클러스터](https://cdn.slidesharecdn.com/ss_thumbnails/212c3-181012011644-thumbnail.jpg?width=640&height=640&fit=bounds)
![[245]Papago Internals: 모델분석과 응용기술 개발](https://cdn.slidesharecdn.com/ss_thumbnails/245papagointernals1-181012045005-thumbnail.jpg?width=640&height=640&fit=bounds)
![[226]NAVER 광고 deep click prediction: 모델링부터 서빙까지](https://cdn.slidesharecdn.com/ss_thumbnails/226naveraddeepclickprediction-181012024116-thumbnail.jpg?width=640&height=640&fit=bounds)
[212] large scale backend service develpment
- 1. Large Scale Backend ServiceDevelopment using Node.js, Docker, and AWS Daniel Kang Sr. Software Engineer / Riot Games [email protected]
- 3. I’ll talk about… •Problems • Solutions • Goals • Node.js, Redis, Docker and AWS • Results
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.
- 29.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40.
- 41.
- 42.
- 43.
- 44.
- 45.
- 46.
- 47.
- 48.
- 49.
- 50.
- 51.
- 52.
- 53.
- 54.
- 55.
- 56.
- 57.
- 58.
- 59.
- 60.
- 61.
- 62.
- 63.
- 64.
- 65.
- 66.
- 67.
- 68.
- 69.
- 70.
- 71.
- 72.
- 73.
- 74.
- 75.
- 76.
- 77.
- 78.
- 79.
- 80.
- 81.
- 82.
- 83.
- 84.
- 85.
- 86.
- 87.
- 88.
- 89.
- 90.
- 91.
- 92.
- 93.
- 94.
- 95.
- 96.
- 97.
- 98.
- 99.
- 100.
- 101.
- 102.
- 103.
- 104.
- 105. Project: Leaderboards • 목표:게임 내 특정 조건을 만족하는 플 레이어들에게 실시간으로 보상 아이템을 주고, 플레이어들이 실시간으로 자신의 진행 상황을 확인 할 수 있는 API도 필요 하다.
- 106.
- 107. Goals • Scalability • Performance •Fast Iterations
- 109.
- 110. Multi-Core Systems • ReverseProxying • Hard to find balanced configs • Tested w/ Nginx and HAProxy • Node.js Native Clustering • Worked pretty well for use cases • Tested on 0.12.x
- 113. CPU Profiling • V8profiling • Node options :“--prof” • Performance Bottlenecks • Unnecessary HTTP routing • Unnecessary encryption (crypto) module usages • DEBUG enabled dependencies • NewRelic Node.js agent: negative impact on performance
- 114. More on Node.js •Node versions: 0.10.x / 0.12.x • ES6 features: generators and block scoping • Linting • Quality of code • Identify potential issues • WebStorm IDE • Code analysis • Interactive debugging features • V8 profiling
- 115.
- 120.
- 122. ECSECS Tasks ECS Services ECSTask Definitions ECS Container Instances ECS Agent EC2 IAM Roles EC2 Instances EC2 Security Groups Elastic Load Balancer CloudWatch Metrics EC2 AMIs VPCs 출처:
- 123.
- 124.
- 135. EC2 as DockerHost • Amazon Linux HVM • Enhanced networking enabled by default • PV not recommended • Larger instances = more performing spec • c4.xlarge (4 cores / 7.5G memory) • T2 class: too much CPU steals • Increase ulimit / nproc • Host: limits.conf, 90-nproc.conf • Docker: --default-ulimit nofile=64000:64000 --default-ulimit nproc=32000:32000
- 136.
- 137.
- 138.
- 139. [c4.4xlarge] $0.928 x100 x 24h x 30d = $66,816 [c4.xlarge] $0.232 x 100 x 24h x 30d = $16,704 saved $50,112
- 140.