xxl job

This document provides an overview of two open-source projects: XXL-JOB, a distributed task scheduling framework, and Mitsuba 3, a research-oriented rendering system. Both projects offer detailed documentation, tutorials, and community support. XXL-JOB focuses on ease of use and scalability, while Mitsuba 3 prioritizes high performance and differentiability in rendering simulations.

XXL-JOB

XXL-JOB, a distributed task scheduling framework.

-- Home Page --

Introduction

XXL-JOB is a distributed task scheduling framework.

It's core design goal is to develop quickly and learn simple, lightweight, and easy to expand.

Now, it's already open source, and many companies use it in production environments, real "out-of-the-box".

XXL-JOB is a distributed task scheduling platform. Its core design goals are rapid development, easy learning, lightweight, and easy to expand. Now the source code is open and connected to the online product lines of many companies, and it can be used out of the box.

Sponsor

XXL-JOB is an open source and free project, with its ongoing development made possible entirely by the support of these awesome backers.

XXL-JOB is an open source and free project, and its ongoing development is entirely possible thanks to the support of its backers. Open source is not easy, go to sponsor project development

Documentation

Communication

Features

Development

In mid-2015, I created the XXL-JOB project warehouse on github and submitted the first commit. Then I carried out system structure design, UI selection, interaction design...

From 2015 to November, XXL-JOB finally RELEASE the first major version V1.0, and then I released it to OSCHINA. XXL-JOB was recommended by @红水 on OSCHINA, and reached OSCHINA’s “ "Hot Moves" ranked first and git.oschina's open source software ranked first in monthly popularity. I would like to express my special thanks to Hongshu and thank you all for your attention and support.

From 2015 to December, I published XXL-JOB to our company's internal knowledge base, and it was recognized by internal colleagues.

From January 2016 to January, our company launched the internal access and customization work of XXL-JOB. We would like to thank two colleagues, Yuan and Yin, for their contribution, and also thank other internal colleagues who have given attention and support.

On 2017-05-13, in the "Launching Up" session of the "62nd Open Source China Source Innovation Conference" held in Shanghai, I took the stage to give a speech on XXL-JOB, and the 500 audience members responded enthusiastically (pictured) article review).

On 2017-10-22, I took part in the "Offensive Microservices Practical School Shanghai Station" organized by Cloud Open Talk and Spring Cloud China Community. I took the stage to give a speech on XXL-JOB. The audience responded enthusiastically and talked with XXL after the meeting. -JOB users have heated discussions and exchanges.

On 2017-12-11, XXL-JOB was honored to participate in the "InfoQ ArchSummit Global Architect Summit" and was introduced on-site by "Teacher Yang Bo", Director of Architecture of Paipaidai, in the special topic "Microservice Principles, Infrastructure and Open Source Practices".

On 2017-12-18, XXL-JOB participated in the "2017 Most Popular Chinese Open Source Software" competition, competing among about 9,000 domestic open source projects that had been entered at that time, and finally entered the top 30.

On January 15, 2018, XXL-JOB participated in the "2017 Code Cloud Most Popular Open Source Projects" competition. It competed among about 6,500 code cloud projects that had been entered at that time, and finally entered the top 20.

On April 14, 2018, at the "2018 Internet Developers Conference" held by iTechPlus in Shanghai, I took the stage to give a speech on XXL-JOB. The audience responded enthusiastically and had a heated discussion with XXL-JOB users after the meeting.

On 2018-05-27, at the "Architecture" theme session of the "75th Open Source China Source Innovation Conference" held in Shanghai, I took the stage to give a keynote speech on "Infrastructure and Middleware Map", and thousands of audience members responded. Enthusiastic (picture and text review).

On 2018-12-05, XXL-JOB participated in the "2018 Most Popular Chinese Open Source Software" competition, competing among more than 10,000 open source projects that had been entered at that time, and finally ranked 19th.

On 2019-12-10, XXL-JOB participated in the "2019 Most Popular Chinese Open Source Software" competition, competing among more than 10,000 open source projects that had been entered at that time, and finally ranked 9th in the "Development Framework and Basic Components Category" .

On November 16, 2020, XXL-JOB participated in the "2020 Most Popular Chinese Open Source Software" competition, competing among more than 10,000 open source projects that had been entered at that time, and finally ranked 8th in the "Development Framework and Basic Components Category" .

On 2021-12-06, XXL-JOB participated in the "2021 OSC China Open Source Project Selection" competition. It competed among more than 10,000 open source projects that had been entered at that time, and was finally selected as the "Most Popular Project".

Our company Dianping has currently accessed XXL-JOB, which has an internal alias of "Ferrari" (Ferrari is customized based on the V1.1 version of XXL-JOB, and it is recommended for new access applications to upgrade to the latest version).

According to the latest statistics, from 2016-01-21 to 2017-12-01, the system has been dispatched approximately 1 million times, showing excellent performance. It is recommended that new access applications use the latest version, because after dozens of version updates, the system's task model, UI interaction model, and underlying scheduling communication model have been greatly optimized and improved, and the core functions are more stable and efficient.

So far, XXL-JOB has been connected to the online product lines of many companies, and the access scenarios include e-commerce business, O2O business and big data operations, etc. As of the latest statistical time, the companies that XXL-JOB has been connected to include but not limited to :

More connected companies are welcome to register at the registration address. Registration is only for product promotion.

Everyone is welcome to pay attention and use, XXL-JOB will also embrace changes and continue to develop.

Contributing

Contributions are welcome! Open a pull request to fix a bug, or open an Issue to discuss a new feature or change.

Welcome to contribute to the project! For example, submit a PR to fix a bug, or create a new Issue to discuss new features or changes.

Copyright and License

This product is open source and free, and will continue to provide free community technical support. Individual or enterprise users are free to access and use.

The product is open source and free, and free community technical support will continue to be provided. It can be freely accessed and used by individuals or enterprises. If necessary, you can contact the author by email to obtain free project authorization.

example:

Mitsuba Renderer 3

Documentation	Tutorial videos	Linux	MacOS	Windows	PyPI

️

Warning

️

There is currently a large amount of undocumented and unstable work going on in

the master branch. We'd highly recommend you use our

latest release

until further notice.

If you already want to try out the upcoming changes, please have a look at

this porting guide.

It should cover most of the new features and breaking changes that are coming.

Introduction

Mitsuba 3 is a research-oriented rendering system for forward and inverse light

transport simulation developed at EPFL in Switzerland.

It consists of a core library and a set of plugins that implement functionality

ranging from materials and light sources to complete rendering algorithms.

Mitsuba 3 is retargetable : this means that the underlying implementations and

data structures can transform to accomplish various different tasks. For

example, the same code can simulate both scalar (classic one-ray-at-a-time) RGB transport

or differential spectral transport on the GPU. This all builds on

Dr.Jit, a specialized just-in-time (JIT) compiler developed specifically for this project.

Main Features

• Cross-platform : Mitsuba 3 has been tested on Linux ( x86_64 ), macOS

  ( aarch64 , x8664 ), and Windows ( x8664 ).

• High performance : The underlying Dr.Jit compiler fuses rendering code

  into kernels that achieve state-of-the-art performance using

  an LLVM backend targeting the CPU and a CUDA/OptiX backend

  targeting NVIDIA GPUs with ray tracing hardware acceleration.

• Python first : Mitsuba 3 is deeply integrated with Python. Materials,

  textures, and even full rendering algorithms can be developed in Python,

  which the system JIT-compiles (and optionally differentiates) on the fly.

  This enables the experimentation needed for research in computer graphics and

  other disciplines.

• Differentiation : Mitsuba 3 is a differentiable renderer, meaning that it

  can compute derivatives of the entire simulation with respect to input

  parameters such as camera pose, geometry, BSDFs, textures, and volumes. It

  implements recent differentiable rendering algorithms developed at EPFL.

• Spectral & Polarization : Mitsuba 3 can be used as a monochromatic

  renderer, RGB-based renderer, or spectral renderer. Each variant can

  optionally account for the effects of polarization if desired.

Tutorial videos, documentation

We've recorded several YouTube videos that provide a gentle introduction

Mitsuba 3 and Dr.Jit. Beyond this you can find complete Juypter notebooks

covering a variety of applications, how-to guides, and reference documentation

on readthedocs.

Installation

We provide pre-compiled binary wheels via PyPI. Installing Mitsuba this way is as simple as running

 pip install mitsuba

on the command line. The Python package includes thirteen variants by default:

• scalar_rgb

• scalar_spectral

• scalarspectralpolarized

• llvmadrgb

• llvmadmono

• llvmadmono_polarized

• llvmadspectral

• llvmadspectral_polarized

• cudaadrgb

• cudaadmono

• cudaadmono_polarized

• cudaadspectral

• cudaadspectral_polarized

The first two perform classic one-ray-at-a-time simulation using either a RGB

or spectral color representation, while the latter two can be used for inverse

rendering on the CPU or GPU. To access additional variants, you will need to

compile a custom version of Dr.Jit using CMake. Please see the

documentation

for details on this.

Requirements

• Python >= 3.8

• (optional) For computation on the GPU: Nvidia driver >= 495.89

• (optional) For vectorized / parallel computation on the CPU: LLVM >= 11.1

Usage

Here is a simple "Hello World" example that shows how simple it is to render a

scene using Mitsuba 3 from Python:

 # Import the library using the alias "mi"import mitsuba as mi# Set the variant of the renderermi.setvariant('scalarrgb')# Load a scenes = mi.loaddict(mi.cornellbox())# Render the sceneimg = mi. render(scene)# Write the rendered image to an EXR filemi.Bitmap(img).write('cbox.exr')

Tutorials and example notebooks covering a variety of applications can be found

in the documentation.

About

This project was created by Wenzel Jakob.

Significant features and/or improvements to the code were contributed by

Sébastien Speierer,

Nicolas Roussel,

Merlin Nimier-David,

Delio Vicini,

Tizian Zeltner,

Baptiste Nicolet,

Miguel Crespo,

Vincent Leroy, and

Ziyi Zhang.

When using Mitsuba 3 in academic projects, please cite:

 @software{Mitsuba3,title = {Mitsuba 3 renderer},author = {Wenzel Jakob and Sébastien Speierer and Nicolas Roussel and Merlin Nimier-David and Delio Vicini and Tizian Zeltner and Baptiste Nicolet and Miguel Crespo and Vincent Leroy and Ziyi Zhang},note = {https://mitsuba-renderer.org},version = {3.1.1},year = 2022}