diff --git a/README.md b/README.md index e84d02d448..33ac08ec1e 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,18 @@ -# Nerfies +# Reinforcement Learning for Automation of High-Level Tasks -This is the repository that contains source code for the [Nerfies website](https://nerfies.github.io). +This is the repository that contains source code for the [Reinforcement Learning for Automation of High-Level Tasks website](https://nineman-yu.github.io). -If you find Nerfies useful for your work please cite: +If you find Reinforcement Learning for Automation of High-Level Tasks useful for your work please cite: ``` -@article{park2021nerfies - author = {Park, Keunhong and Sinha, Utkarsh and Barron, Jonathan T. and Bouaziz, Sofien and Goldman, Dan B and Seitz, Steven M. and Martin-Brualla, Ricardo}, - title = {Nerfies: Deformable Neural Radiance Fields}, - journal = {ICCV}, - year = {2021}, +@article{kwon2024reinforcement, + title={Reinforcement Learning with Task Decomposition and Task-Specific Reward System for Automation of High-Level Tasks}, + author={Kwon, Gunam and Kim, Byeongjun and Kwon, Nam Kyu}, + journal={Biomimetics}, + volume={9}, + number={4}, + pages={196}, + year={2024}, + publisher={MDPI} } ``` diff --git a/index.html b/index.html index 373119fe36..f55b08be31 100644 --- a/index.html +++ b/index.html @@ -3,10 +3,10 @@ - + content="Reinforcement Learning with Task Decomposition and Task-Specific Reward System for Automation of High-Level Tasks."> + - Nerfies: Deformable Neural Radiance Fields + Reinforcement Learning with Task Decomposition and Task-Specific Reward System for Automation of High-Level Tasks @@ -32,7 +32,7 @@ - + @@ -52,7 +52,7 @@
- + @@ -63,17 +63,8 @@ More Research
- - HyperNeRF - - - Nerfies - - - LatentFusion - - - PhotoShape + + Task_decomposition(Pick-and-Place)
@@ -88,39 +79,26 @@
-

Nerfies: Deformable Neural Radiance Fields

+

Reinforcement Learning with Task Decomposition and Task-Specific Reward System for Automation of High-Level Tasks

- Keunhong Park1, - - Utkarsh Sinha2, + Gunam Kwon1, - Jonathan T. Barron2, - - - Sofien Bouaziz2, - + Byeongjun Kim1, - Dan B Goldman2, - - - Steven M. Seitz1,2, - - - Ricardo Martin-Brualla2 + Nam Kyu Kwon1,
- 1University of Washington, - 2Google Research + 1Yeungnam University,
- @@ -129,15 +107,14 @@

Nerfies: Deformable Neural Radiance Fie - + - + MDPI Icon - arXiv + MDPI - + + - @@ -157,15 +135,6 @@

Nerfies: Deformable Neural Radiance Fie Code - - - - - - - Data -

@@ -175,75 +144,17 @@

Nerfies: Deformable Neural Radiance Fie

+
-
- -

- Nerfies turns selfie videos from your phone into - free-viewpoint - portraits. -

-
-
-
- - -
-
-
- @@ -258,197 +169,106 @@

Abstract

- We present the first method capable of photorealistically reconstructing a non-rigidly - deforming scene using photos/videos captured casually from mobile phones. + We provide a method for task decomposition and task-specific rewards for high-level tasks.

- Our approach augments neural radiance fields - (NeRF) by optimizing an - additional continuous volumetric deformation field that warps each observed point into a - canonical 5D NeRF. - We observe that these NeRF-like deformation fields are prone to local minima, and - propose a coarse-to-fine optimization method for coordinate-based models that allows for - more robust optimization. - By adapting principles from geometry processing and physical simulation to NeRF-like - models, we propose an elastic regularization of the deformation field that further - improves robustness. -

-

- We show that Nerfies can turn casually captured selfie - photos/videos into deformable NeRF - models that allow for photorealistic renderings of the subject from arbitrary - viewpoints, which we dub "nerfies". We evaluate our method by collecting data - using a - rig with two mobile phones that take time-synchronized photos, yielding train/validation - images of the same pose at different viewpoints. We show that our method faithfully - reconstructs non-rigidly deforming scenes and reproduces unseen views with high - fidelity. + This paper introduces a reinforcement learning method that leverages task decomposition + and a task-specific reward system to address complex high-level tasks, such as door opening, block + stacking, and nut assembly. These tasks are decomposed into various subtasks, with the grasping and + putting tasks executed through single joint and gripper actions, while other tasks are trained using + the SAC algorithm alongside the task-specific reward system. The task-specific reward system aims + to increase the learning speed, enhance the success rate, and enable more efficient task execution. + The experimental results demonstrate the efficacy of the proposed method, achieving success rates + of 99.9% for door opening, 95.25% for block stacking, 80.8% for square-nut assembly, and 90.9% for + round-nut assembly. Overall, this method presents a promising solution to address the challenges + associated with complex tasks, offering improvements over the traditional end-to-end approach.

- - -
-
-

Video

-
- -
-
-
-
- - +
-
- -
- - -
-
-

Visual Effects

-

- Using nerfies you can create fun visual effects. This Dolly zoom effect - would be impossible without nerfies since it would require going through a wall. -

- -
-
- - - -
-

Matting

+
+
-
-

- As a byproduct of our method, we can also solve the matting problem by ignoring - samples that fall outside of a bounding box during rendering. -

- -
- -
-
-
- - - -
-
-

Animation

- - -

Interpolating states

-
-

- We can also animate the scene by interpolating the deformation latent codes of two input - frames. Use the slider here to linearly interpolate between the left frame and the right - frame. -

-
-
-
- Interpolate start reference image. -

Start Frame

-
-
-
- Loading... + +
+
+

Door-opening task

+

The door-opening task is decomposed into reaching, grasping, turning, and pulling, where all tasks except grasping are trained using SAC(Soft Actor-Critic) and performed sequentially.

+ +
+
+ +
+
+

Block-stacking task

+

The block-stacking task is decomposed into reaching, grasping, reaching, and putting, where all tasks except grasping and putting are trained using SAC and performed sequentially.

+ +
- -
-
- Interpolation end reference image. -

End Frame

-
-
-
- - - -

Re-rendering the input video

-
-

- Using Nerfies, you can re-render a video from a novel - viewpoint such as a stabilized camera by playing back the training deformations. -

-
- + +
+ +
+
+

Round-nut assembly task

+

The nut-assembly task is decomposed into reaching, aligning, reaching, grasping, assembly, and putting, where all tasks except grasping and putting are trained using SAC and performed sequentially.

+ +
+
+ +
+
+

Square-nut assembly task

+

The nut-assembly task is decomposed into reaching, aligning, reaching, grasping, assembly, and putting, where all tasks except grasping and putting are trained using SAC and performed sequentially.

+ +
+
- - -
-
- - - - -
-
-

Related Links

- -
-

- There's a lot of excellent work that was introduced around the same time as ours. -

-

- Progressive Encoding for Neural Optimization introduces an idea similar to our windowed position encoding for coarse-to-fine optimization. -

-

- D-NeRF and NR-NeRF - both use deformation fields to model non-rigid scenes. -

-

- Some works model videos with a NeRF by directly modulating the density, such as Video-NeRF, NSFF, and DyNeRF -

-

- There are probably many more by the time you are reading this. Check out Frank Dellart's survey on recent NeRF papers, and Yen-Chen Lin's curated list of NeRF papers. -

+ +
+
+

Related Links

+
+

There's a lot of excellent work that was introduced around the same time as ours.

+

+ The Task Decomposition and Dedicated Reward-System-Based Reinforcement Learning Algorithm for Pick-and-Place serves as the foundation of the methodology outlined in the paper 'Reinforcement Learning with Task Decomposition and Task-Specific Reward System for Automation of High-Level Tasks. +

+

+ Towards Hierarchical Task Decomposition using Deep Reinforcement Learning for Pick and Place Subtasks introduces pick-and-place divided into approach, manipulate, and retract, and successfully performs pick-and-place using DDPG and HER. +

+
+
-
+
- - -
- - +

BibTeX

- 
@article{park2021nerfies,
-  author    = {Park, Keunhong and Sinha, Utkarsh and Barron, Jonathan T. and Bouaziz, Sofien and Goldman, Dan B and Seitz, Steven M. and Martin-Brualla, Ricardo},
-  title     = {Nerfies: Deformable Neural Radiance Fields},
-  journal   = {ICCV},
-  year      = {2021},
+    
@article{kwon2024reinforcement,
+  title={Reinforcement Learning with Task Decomposition and Task-Specific Reward System for Automation of High-Level Tasks},
+  author={Kwon, Gunam and Kim, Byeongjun and Kwon, Nam Kyu},
+  journal={Biomimetics},
+  volume={9},
+  number={4},
+  pages={196},
+  year={2024},
+  publisher={MDPI}
 }
@@ -458,10 +278,10 @@

BibTeX

+ href="./static/videos/high-level_paper.pdf"> - +
diff --git a/static/images/ICAS_logo.png b/static/images/ICAS_logo.png new file mode 100644 index 0000000000..ffd173b16e Binary files /dev/null and b/static/images/ICAS_logo.png differ diff --git a/static/images/interpolate_end.jpg b/static/images/interpolate_end.jpg deleted file mode 100644 index 7792aabe65..0000000000 Binary files a/static/images/interpolate_end.jpg and /dev/null differ diff --git a/static/images/interpolate_start.jpg b/static/images/interpolate_start.jpg deleted file mode 100644 index 1180958ecb..0000000000 Binary files a/static/images/interpolate_start.jpg and /dev/null differ diff --git a/static/images/mdpi_logo.png b/static/images/mdpi_logo.png new file mode 100644 index 0000000000..9f203384d2 Binary files /dev/null and b/static/images/mdpi_logo.png differ diff --git a/static/images/steve.webm b/static/images/steve.webm deleted file mode 100644 index 9017a1d347..0000000000 Binary files a/static/images/steve.webm and /dev/null differ diff --git a/static/videos/block_stacking_task.mp4 b/static/videos/block_stacking_task.mp4 new file mode 100644 index 0000000000..c66c9390c3 Binary files /dev/null and b/static/videos/block_stacking_task.mp4 differ diff --git a/static/videos/blueshirt.mp4 b/static/videos/blueshirt.mp4 deleted file mode 100644 index 12c3c16095..0000000000 Binary files a/static/videos/blueshirt.mp4 and /dev/null differ diff --git a/static/videos/book.mp4 b/static/videos/book.mp4 deleted file mode 100644 index adb6e3cbb1..0000000000 Binary files a/static/videos/book.mp4 and /dev/null differ diff --git a/static/videos/chair-tp.mp4 b/static/videos/chair-tp.mp4 deleted file mode 100644 index ac2f0a557c..0000000000 Binary files a/static/videos/chair-tp.mp4 and /dev/null differ diff --git a/static/videos/coffee.mp4 b/static/videos/coffee.mp4 deleted file mode 100644 index 9e07095d8f..0000000000 Binary files a/static/videos/coffee.mp4 and /dev/null differ diff --git a/static/videos/dollyzoom-depth.mp4 b/static/videos/dollyzoom-depth.mp4 deleted file mode 100644 index 1578788d22..0000000000 Binary files a/static/videos/dollyzoom-depth.mp4 and /dev/null differ diff --git a/static/videos/dollyzoom-stacked.mp4 b/static/videos/dollyzoom-stacked.mp4 deleted file mode 100644 index b22a749080..0000000000 Binary files a/static/videos/dollyzoom-stacked.mp4 and /dev/null differ diff --git a/static/videos/dollyzoom.mp4 b/static/videos/dollyzoom.mp4 deleted file mode 100644 index 12be1a5278..0000000000 Binary files a/static/videos/dollyzoom.mp4 and /dev/null differ diff --git a/static/videos/door_opening_task.mp4 b/static/videos/door_opening_task.mp4 new file mode 100644 index 0000000000..d64150cb91 Binary files /dev/null and b/static/videos/door_opening_task.mp4 differ diff --git a/static/videos/fullbody.mp4 b/static/videos/fullbody.mp4 deleted file mode 100644 index af5c4ebec9..0000000000 Binary files a/static/videos/fullbody.mp4 and /dev/null differ diff --git a/static/videos/high-level_paper.pdf b/static/videos/high-level_paper.pdf new file mode 100644 index 0000000000..ce48215698 Binary files /dev/null and b/static/videos/high-level_paper.pdf differ diff --git a/static/videos/mask-nobg.mp4 b/static/videos/mask-nobg.mp4 deleted file mode 100644 index 8a68807605..0000000000 Binary files a/static/videos/mask-nobg.mp4 and /dev/null differ diff --git a/static/videos/mask.mp4 b/static/videos/mask.mp4 deleted file mode 100644 index fc92c5c9b8..0000000000 Binary files a/static/videos/mask.mp4 and /dev/null differ diff --git a/static/videos/matting.mp4 b/static/videos/matting.mp4 deleted file mode 100644 index 5620b18bca..0000000000 Binary files a/static/videos/matting.mp4 and /dev/null differ diff --git a/static/videos/replay.mp4 b/static/videos/replay.mp4 deleted file mode 100644 index f2fa33e920..0000000000 Binary files a/static/videos/replay.mp4 and /dev/null differ diff --git a/static/videos/round_nut_assembly_task.mp4 b/static/videos/round_nut_assembly_task.mp4 new file mode 100644 index 0000000000..874fb3a8ba Binary files /dev/null and b/static/videos/round_nut_assembly_task.mp4 differ diff --git a/static/videos/shiba.mp4 b/static/videos/shiba.mp4 deleted file mode 100644 index 3833f2317c..0000000000 Binary files a/static/videos/shiba.mp4 and /dev/null differ diff --git a/static/videos/square_nut_assembly_task.mp4 b/static/videos/square_nut_assembly_task.mp4 new file mode 100644 index 0000000000..30a976ad81 Binary files /dev/null and b/static/videos/square_nut_assembly_task.mp4 differ diff --git a/static/videos/steve.mp4 b/static/videos/steve.mp4 deleted file mode 100644 index ae949f7779..0000000000 Binary files a/static/videos/steve.mp4 and /dev/null differ diff --git a/static/videos/teaser.mp4 b/static/videos/teaser.mp4 deleted file mode 100644 index b891a08ff7..0000000000 Binary files a/static/videos/teaser.mp4 and /dev/null differ diff --git a/static/videos/toby.mp4 b/static/videos/toby.mp4 deleted file mode 100644 index 307f027bd6..0000000000 Binary files a/static/videos/toby.mp4 and /dev/null differ diff --git a/static/videos/toby2.mp4 b/static/videos/toby2.mp4 deleted file mode 100644 index 483b36c2b2..0000000000 Binary files a/static/videos/toby2.mp4 and /dev/null differ diff --git a/static/videos/total_tasks.mp4 b/static/videos/total_tasks.mp4 new file mode 100644 index 0000000000..ba44498655 Binary files /dev/null and b/static/videos/total_tasks.mp4 differ