You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Lithops is a Python multi-cloud distributed computing framework. It allows you to run unmodified local python code at massive scale in the main
20
-
serverless computing platforms. Lithops delivers the user’s code into the cloud without requiring knowledge of how it is deployed and run. Moreover, its multicloud-agnostic architecture ensures portability across cloud providers.
21
-
22
-
Lithops is specially suited for highly-parallel programs with little or no need for communication between processes, but it also supports parallel applications that need to share state among processes. Examples of applications that run with Lithops include Monte Carlo simulations, deep learning and machine learning processes, metabolomics computations, and geospatial analytics, to name a few.
11
+
Lithops is a Python multi-cloud distributed computing framework that lets you run unmodified Python code at massive scale across cloud, HPC, and on-premise platforms. It supports major cloud providers and Kubernetes platforms, running your code transparently without requiring you to manage deployment or infrastructure.
23
12
13
+
Lithops is ideal for highly parallel workloads—such as Monte Carlo simulations, machine learning, metabolomics, or geospatial analytics—and lets you tailor execution to your priorities: you can optimize for performance using AWS Lambda to launch hundreds of functions in milliseconds, or reduce costs by running the same code on AWS Batch with Spot Instances.
24
14
25
15
## Installation
26
16
@@ -37,7 +27,7 @@ Lithops is specially suited for highly-parallel programs with little or no need
37
27
```
38
28
39
29
## Configuration
40
-
Lithops provides an extensible backend architecture (compute, storage) that is designed to work with different Cloud providers and on-premise backends. In this sense, you can code inpython and run it unmodified in IBM Cloud, AWS, Azure, Google Cloud, Aliyun and Kubernetes or OpenShift.
30
+
Lithops provides an extensible backend architecture (compute, storage) designed to work with various cloud providers and on-premise platforms. You can write your code inPython and run it unmodified across major cloud providers and Kubernetes environments.
41
31
42
32
[Follow these instructions to configure your compute and storage backends](config/)
43
33
@@ -71,11 +61,11 @@ Lithops is shipped with 2 different high-level Compute APIs, and 2 high-level St
71
61
```python
72
62
from lithops import FunctionExecutor
73
63
74
-
def hello(name):
75
-
returnf'Hello {name}!'
64
+
def double(i):
65
+
returni * 2
76
66
77
67
with FunctionExecutor() as fexec:
78
-
f = fexec.call_async(hello, 'World')
68
+
f = fexec.map(double, [1, 2, 3, 4])
79
69
print(f.result())
80
70
```
81
71
</td>
@@ -146,23 +136,6 @@ if __name__ == "__main__":
146
136
147
137
You can find more usage examples in the [examples](/examples) folder.
148
138
149
-
## Execution Modes
150
-
151
-
Lithops is shipped with 3 different modes of execution. The execution mode allows you to decide where and how the functions are executed.
This mode allows you to execute functions on the local machine using processes, providing a convenient way to leverage Lithops' distributed computing capabilities without relying on cloud resources. This mode is particularly useful for development, testing, and debugging purposes. This is the default mode of execution if no configuration is provided.
This mode allows you to execute functions on popular serverless compute services, leveraging the scalability, isolation, and automatic resource provisioning provided by these platforms. With serverless mode, you can easily parallelize task execution, harness the elastic nature of serverless environments, and simplify the development and deployment of scalable data processing workloads and parallel applications.
This mode provides the capability to execute functions on one or multiple virtual machines (VMs) simultaneously, in a serverless-like fashion, without requiring manual provisioning as everything is automatically created. This mode can be used in a private cluster or in the cloud, where functions within each VM are executed using parallel processes.
164
-
165
-
166
139
## Documentation
167
140
168
141
For documentation on using Lithops, see [latest release documentation](https://lithops-cloud.github.io/docs/) or [current github docs](docs/user_guide.md).
@@ -174,15 +147,15 @@ If you are interested in contributing, see [CONTRIBUTING.md](./CONTRIBUTING.md).
174
147
### Blogs and Talks
175
148
* [Simplify the developer experience with OpenShift for Big Data processing by using Lithops framework](https://medium.com/@gvernik/simplify-the-developer-experience-with-openshift-for-big-data-processing-by-using-lithops-framework-d62a795b5e1c)
176
149
* [Speed-up your Python applications using Lithops and Serverless Cloud resources](https://itnext.io/speed-up-your-python-applications-using-lithops-and-serverless-cloud-resources-a64beb008bb5)
177
-
* [Serverless Without Constraints](https://www.ibm.com/cloud/blog/serverless-without-constraints)
150
+
* [Serverless Without Constraints](https://www.ibm.com/blog/serverless-without-constraints)
178
151
* [Lithops, a Multi-cloud Serverless Programming Framework](https://itnext.io/lithops-a-multi-cloud-serverless-programming-framework-fd97f0d5e9e4)
* [Using Serverless to Run Your Python Code on 1000 Cores by Changing Two Lines of Code](https://www.ibm.com/cloud/blog/using-serverless-to-run-your-python-code-on-1000-cores-by-changing-two-lines-of-code)
181
-
* [Decoding dark molecular matter in spatial metabolomics with IBM Cloud Functions](https://www.ibm.com/cloud/blog/decoding-dark-molecular-matter-in-spatial-metabolomics-with-ibm-cloud-functions)
153
+
* [Using Serverless to Run Your Python Code on 1000 Cores by Changing Two Lines of Code](https://www.ibm.com/blog/using-serverless-to-run-your-python-code-on-1000-cores-by-changing-two-lines-of-code)
154
+
* [Decoding dark molecular matter in spatial metabolomics with IBM Cloud Functions](https://www.ibm.com/blog/decoding-dark-molecular-matter-in-spatial-metabolomics-with-ibm-cloud-functions)
182
155
* [Your easy move to serverless computing and radically simplified data processing](https://www.slideshare.net/gvernik/your-easy-move-to-serverless-computing-and-radically-simplified-data-processing-238929020) Strata Data Conference, NY 2019. See video of Lithops usage [here](https://www.youtube.com/watch?v=EYa95KyYEtg&list=PLpR7f3Www9KCjYisaG7AMaR0C2GqLUh2G&index=3&t=0s) and the example of Monte Carlo [here](https://www.youtube.com/watch?v=vF5HI2q5VKw&list=PLpR7f3Www9KCjYisaG7AMaR0C2GqLUh2G&index=2&t=0s)
183
156
* [Speed up data pre-processing with Lithops in deep learning](https://developer.ibm.com/patterns/speed-up-data-pre-processing-with-pywren-in-deep-learning/)
184
-
* [Predicting the future with Monte Carlo simulations over IBM Cloud Functions](https://www.ibm.com/cloud/blog/monte-carlo-simulations-with-ibm-cloud-functions)
185
-
* [Process large data sets at massive scale with Lithops over IBM Cloud Functions](https://www.ibm.com/cloud/blog/process-large-data-sets-massive-scale-pywren-ibm-cloud-functions)
157
+
* [Predicting the future with Monte Carlo simulations over IBM Cloud Functions](https://www.ibm.com/blog/monte-carlo-simulations-with-ibm-cloud-functions)
158
+
* [Process large data sets at massive scale with Lithops over IBM Cloud Functions](https://www.ibm.com/blog/process-large-data-sets-massive-scale-pywren-ibm-cloud-functions)
186
159
* [Industrial project in Technion on Lithops](http://www.cs.technion.ac.il/~cs234313/projects_sites/W19/04/site/)
0 commit comments