Merge pull request #1 from orgicus/develop

WIP to master branch

.classpath

@@ -2,5 +2,9 @@
 <classpath>
 	<classpathentry kind="src" output="bin" path="src"/>
 	<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
+	<classpathentry kind="lib" path="/Users/George/Downloads/ProcessingDNLD/Processing3.5.3.app/Contents/Java/core.jar"/>
+	<classpathentry kind="lib" path="/Users/George/Documents/Processing3/libraries/oscP5/library/oscP5.jar"/>
+	<classpathentry kind="lib" path="lib/okhttp.jar"/>
+	<classpathentry kind="lib" path="lib/okio.jar"/>
 	<classpathentry kind="output" path="resources/code"/>
 </classpath>

README.md

@@ -1,100 +1,133 @@
-The following describes how to set up a Processing Library project in Eclipse and build it successfully, and to make your Library ready for distribution.
-
-## Import to Eclipse
-
-There are two options to import the template project into Eclipse: using a Git [fork](https://help.github.com/articles/fork-a-repo) or using a downloaded package. If you are not familiar with Git or GitHub, you should opt for the downloaded package.
-
-### Option A: GitHub
-
-1. Fork the template repository to use as a starting point.
-  * Navigate to https://github.com/processing/processing-library-template in your browser.
-  * Click the "Fork" button in the top-right of the page.
-  * Once your fork is ready, open the new repository's "Settings" by clicking the link in the menu bar on the right.
-  * Change the repository name to the name of your Library and save your changes.
-  * NOTE: GitHub only allows you to fork a project once. If you need to create multiple forks, you can follow these [instructions](https://beeznest.wordpress.com/2013/12/26/creating-multiple-forks-using-upstream-branches/).
-1. Clone your new repository to your Eclipse workspace.
-  * Open Eclipse and select the File → Import... menu item.
-  * Select Git → Projects from Git, and click "Next >".
-  * Select "URI" and click "Next >". 
-  * Enter your repository's clone URL in the "URI" field. The remaining fields in the "Location" and "Connection" groups will get automatically filled in.
-  * Enter your GitHub credentials in the "Authentication" group, and click "Next >".
-  * Select the `master` branch on the next screen, and click "Next >".
-  * The default settings on the "Local Configuration" screen should work fine, click "Next >".
-  * Make sure "Import existing projects" is selected, and click "Next >".
-  * Eclipse should find and select the `processing-library-template` automatically, click "Finish".
-1. Rename your Eclipse project.
-  * In the Package Explorer, right-click (ctrl-click) on the folder icon of the `processing-library-template` project, and select Refactor → Rename... from the menu that pops up. 
-  * Give the project the name of your Library, and click "OK".
+# RunwayML library for Processing
+
+A library to easily use [RunwayML](https://runwayml.com) with [Processing](https://processing.org/).
+
+## How install 
+
+### Option 1: via Contribution Manager
+
+In Processing:
+
+1. go to **Sketch > Import Library... > Add Library... > Contribution Manager Filter > Runway**
+2. select RunwayML
+3. press install
+
+### Option 2: manually
+
+1. Download **RunwayML.zip** from the [releases page](https://github.com/runwayml/processing-library/releases)
+2. Unzip into **Documents > Processing > libraries**
+3. Restart Processing (if it was already running)
+
+## Basic Examples
+
+This example will print `/data` from Runway (e.g. running im2txt)
+
+```processing
+// import Runway library
+import com.runwayml.*;
+// reference to runway instance
+RunwayHTTP runway;
+
+void setup() {
+  // setup Runway
+  runway = new RunwayHTTP(this);
+}
+
+void draw() {
   
-### Option B: Downloaded Package
+}
+
+// this is called when new Runway data is available
+void runwayDataEvent(JSONObject runwayData){
+  println(runwayData);
+}
+```
 
-1. Download the latest Eclipse template from [here](https://github.com/processing/processing-library-template/releases). **Don't unzip the ZIP file yet.**
-1. Create a new Java project in Eclipse. 
-  * From the menubar choose File → New → Java Project. 
-  * Give the project the name of your Library. 
-  * Click "Finish".
-1. Import the template source files.
-  * Right-click (ctrl-click) onto the folder icon of your newly created project in the Package Explorer and select "Import..." from the menu that pops up. 
-  * Select General → Archive File, and click "Next >".
-  * Navigate to the ZIP file you downloaded earlier in step 1, and click "Finish".
+## Existing Examples
 
-## Set Up and Compile
+You can access the following examples via **Processing > Examples > Contributed Libraries > RunwayML**
 
-1. Add Processing to the project build path.
-  * Open your project's "Properties" window. 
-  * Under "Java Build Path", select the "Libraries" tab and then "Add External JARs...". 
-  * Locate and add Processing's `core.jar` to your build path. It is recommended that a copy of `core.jar` is located in your Eclipse workspace in a `libs` folder. If the `libs` folder does not exist yet, create it. Read the [section below](#AddingJARs) regarding where to find the `core.jar` file.
-  * Confirm the setup with "OK".
-1. Edit the Library properties.
-  * Open the `resources` folder inside of your Java project and double-click the `build.properties` file. You should see its contents in the Eclipse editor. 
-  * Edit the properties file, making changes to items 1-4 so that the values and paths are properly set for your project to compile. A path can be relative or absolute.
-  * Make changes to items under 5. These are metadata used in the automatically generated HTML, README, and properties documents.
-1. Compile your Library using Ant.
-  * From the menu bar, choose Window → Show View → Ant. A tab with the title "Ant" will pop up on the right side of your Eclipse editor. 
-  * Drag the `resources/build.xml` file in there, and a new item "ProcessingLibs" will appear. 
-  * Press the "Play" button inside the "Ant" tab.
-1. BUILD SUCCESSFUL. The Library template will start to compile, control messages will appear in the console window, warnings can be ignored. When finished it should say BUILD SUCCESSFUL. Congratulations, you are set and you can start writing your own Library by making changes to the source code in folder `src`.
-1. BUILD FAILED. In case the compile process fails, check the output in the console which will give you a closer idea of what went wrong. Errors may have been caused by
-  * Incorrect path settings in the `build.properties` file.
-  * Error "Javadoc failed". if you are on Windows, make sure you are using a JDK instead of a JRE in order to be able to create the Javadoc for your Library. JRE does not come with the Javadoc application, but it is required to create Libraries from this template.
+- AdaIN-Style-Transfer ![preview](assets/examples/ada-in-style-transfer.jpg)
+
+- Adaptive-Style-Transfer ![preview](assets/examples/adaptive-style-transfer.jpg)
+
+- Arbitrary-Image-Stylization ![preview](assets/examples/arbitrary-image-stylization.jpg)
+
+- AttnGAN:![preview](assets/examples/attn-gan.jpg)
+
+- BigGAN: ![preview](assets/examples/big-gan.jpg)
 
-After having compiled and built your project successfully, you should be able to find your Library in Processing's sketchbook folder, examples will be listed in Processing's sketchbook menu. Files that have been created for the distribution of the Library are located in your Eclipse's `workspace/yourProject/distribution` folder. In there you will also find the `web` folder which contains the documentation, a ZIP file for downloading your Library, a folder with examples as well as the `index.html` and CSS file.
+- cnOCR: ![preview](assets/examples/cn-ocr.jpg)
 
-To distribute your Library please refer to the [Library Guidelines](https://github.com/processing/processing/wiki/Library-Guidelines).
+- COCO-SSD: ![preview](assets/examples/coco-ssd.jpg)
 
-## Source code
+- CycleGAN: ![preview](assets/examples/cycle-gan.jpg)
 
-If you want to share your Library's source code, we recommend using an online repository available for free at [GitHub](https://github.com/).
+- DeepLab: ![preview](assets/examples/deep-lab.jpg)
 
-## <a name='AddingJARs'/>Adding core.jar and other .jar files to your classpath</a>
+- DenseCap: ![preview](assets/examples/dense-cap.jpg)
 
-The `core.jar` file contains the core classes of Processing and has to be part of your classpath when building a Library. On Windows and Linux, this file is located in the Processing distribution folder inside a folder named `lib`. On Mac OS X, right-click the Processing.app and use "Show Package Contents" to see the guts. The `core.jar` file is inside Contents → Resources → Java. For further information about the classes in `core.jar`, you can see the source [here](http://code.google.com/p/processing/source/browse/trunk/processing#processing/core) and the developer documentation [here](http://processing.googlecode.com/svn/trunk/processing/build/javadoc/core/index.html).
+- DenseDepth: ![preview](assets/examples/dense-depth.jpg)
 
-If you created a `libs` folder as described above, put the libraries you need to add to your classpath in there. In the "Properties" of your Java project, navigate to Java Build Path → Libraries, and click "Add External JARs...". Select the `.jar` files from the `libs` folder that are required for compiling your project. Adjust the `build.xml` file accordingly.
+- DeOldify: ![preview](assets/examples/de-oldify.jpg)
 
-The `libs` folder is recommended but not a requirement, nevertheless you need to specify where your `.jar` files are located in your system in order to add them to the classpath.
+- Face-Landmarks: ![preview](assets/examples/face-landmarks.jpg)
 
-In case a Library depends on system libraries, put these dependencies next to the `.jar` file. For example, Processing's `opengl.jar` Library depends on JOGL hence the DLLs (for Windows) or jnilibs (for OS X) have to be located next to the `opengl.jar` file.
+- Face-Recognition: ![preview](assets/examples/face-recognition)
+
+- Fast-Style-Transfer: ![preview](assets/examples/fast-style-transfer.jpg)
+
+- GPT-2: ![preview](assets/examples/gpt-2.jpg)
+
+- im2txt: ![preview](assets/examples/im2txt.jpg)
+
+- Image-Inpainting-GMCNN: ![preview](assets/examples/inpainting-gmcnn.jpg)
+
+- Image-Super-Resolution: ![preview](assets/examples/image-super-resolution.jpg)
+
+- MaskRCNN: ![preview](assets/examples/mask-rcnn.jpg)
+
+- Model3DDFA: ![preview](assets/examples/3ddfa.jpg)
+
+- OpenPifPaf-Pose: ![preview](assets/examples/open-pif-paf.jpg)
+
+- Photo-Sketch: ![preview](assets/examples/photo-sketch.jpg)
+
+- PoseNet: ![preview](assets/examples/posenet.jpg)
+
+- SPADE-COCO: ![preview](assets/examples/spade-coco.jpg)
+
+- SPADE-Landscapes: ![preview](assets/examples/spade-landscapes.jpg)
+
+- StyleGAN: ![preview](assets/examples/style-gan.jpg)
+
+- VisualImportance: ![preview](assets/examples/visual-importance.jpg)
+
+- YOLACT: ![preview](assets/examples/yolact.jpg)
+
 
-## What is the difference between JDK and JRE?
+## Dependencies
 
-JDK stands for Java Development Kit whereas JRE stands for Java Runtime Environment. For developers it is recommended to work with a JDK instead of a JRE since more Java development related applications such as Javadoc are included. Javadoc is a requirement to properly compile and document a Processing Library as described on the guidelines page.
+This Processing Library manages the OSC connection to Runway relying on the [oscP5](http://www.sojamo.de/libraries/oscP5/) library.
 
-You can have both a JDK and a JRE installed on your system. In Eclipse you need to specify which one you want to use.
+Please install oscP5 before using the OSC connection with this library
 
-## The JRE System Library
+## Contributing
 
-This primarily affects Windows and Linux users (because the full JDK is installed by default on Mac OS X). It is recommended that you use the JDK instead of a JRE. The JDK can be downloaded from [Oracle's download site](http://www.oracle.com/technetwork/java/javase/downloads/index.html). Also see the [Java Platform Installation page](http://www.oracle.com/technetwork/java/javase/index-137561.html), which contains useful information.
+This is still a work in progress. Contributions are welcomed!
 
-To change the JRE used to compile your Java project:
+## Credits
 
-1. Open the properties of your project from the menu Project → Properties. Select "Java Build Path" and in its submenu, click on the "Libraries" tab.
-1. A list of JARs and class folders in the build path will show up. In this list you can find the JRE System Library that is used to compile your code. Remove this JRE System library.
-1. Click "Add Library...". In the popup window, choose "JRE System Library" and press "Next".
-1. Select an alternate JRE from the pull-down menu or click and modify the "Installed JREs". Confirm with "Finish" and "OK". 
+Special thanks for mentoring and support from [Cris Valenzuela, Anastasis Germanidis](https://runwayml.com) and [Daniel Shiffman](https://github.com/shiffman)
 
-## Compiling with Ant and javadoc
+Main library developement by [George Profenza](https://github.com/orgicus)
 
-Ant is a Java-based build tool. For [more information](http://ant.apache.org/faq.html#what-is-ant) visit the [Ant web site](http://ant.apache.org/). Ant uses a file named `build.xml` to store build settings for a project.
+Library examples are based partially on [Runway Processing Examples](https://github.com/runwayml/processing) by:
 
-Javadoc is an application that creates an HTML-based API documentation of Java code. You can check for its existence by typing `javadoc` on the command line. On Mac OS X, it is installed by default. On Windows and Linux, installing the JDK will also install the Javadoc tool. 
+- [Cris Valenzuela](https://github.com/cvalenzuela)
+- [Anastasis Germanidis](https://github.com/agermanidis)
+- [Gene Kogan](https://github.com/genekogan)
+- [maybay21](https://github.com/maybay21)
+- [JP Yepez](https://github.com/jpyepez)
+- [Joel Matthys](https://github.com/jwmatthys)
+- [Alejandro Matamala Ortiz](https://github.com/matamalaortiz)

examples/HTTP/AdaINStyleTransfer/AdaINStyleTransfer.pde

@@ -0,0 +1,157 @@
+// Copyright (C) 2018 Runway AI Examples
+// 
+// This file is part of Runway AI Examples.
+// 
+// Runway-Examples is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+// 
+// Runway-Examples is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+// 
+// You should have received a copy of the GNU General Public License
+// along with RunwayML.  If not, see <http://www.gnu.org/licenses/>.
+// 
+// ===========================================================================
+
+// RUNWAY
+// www.runwayapp.ai
+
+// AdaIN-Style-Transfer
+// Receive HTTP messages from Runway
+// Running AdaIN-Style-Transfer model
+// example by George Profenza
+
+// import video library
+import processing.video.*;
+
+// import Runway library
+import com.runwayml.*;
+// reference to runway instance
+RunwayHTTP runway;
+
+PImage runwayResult;
+
+// periocally to be updated using millis()
+int lastMillis;
+// how often should the above be updated and a time action take place ?
+int waitTime = 1000;
+
+// reference to the camera
+Capture camera;
+
+// status
+String status = "waiting ~"+(waitTime/1000)+"s";
+
+PImage styleImage;
+
+void setup(){
+  // match sketch size to default model camera setup
+  size(1800,400);
+  // setup Runway
+  runway = new RunwayHTTP(this);
+  // update manually
+  runway.setAutoUpdate(false);
+  // setup camera
+  camera = new Capture(this,640,480);
+  camera.start();
+  // setup timer
+  lastMillis = millis();
+}
+
+void draw(){
+  background(0);
+  // update timer
+  int currentMillis = millis();
+  // if the difference between current millis and last time we checked past the wait time
+  if(currentMillis - lastMillis >= waitTime){
+    status = "sending image to Runway";
+    // call the timed function
+    sendFrameToRunway();
+    // update lastMillis, preparing for another wait
+    lastMillis = currentMillis;
+  }
+  // draw style image (if loaded)
+  if(styleImage != null){
+    image(styleImage,600,0);
+  }
+  // draw image received from Runway
+  if(runwayResult != null){
+    image(runwayResult,1200,0);
+  }
+  // draw camera feed
+  image(camera,0,0,600,400);
+  // display status
+  text("Press SPACE to select a style image\n"+status,5,15);
+}
+
+void keyPressed(){
+  if(key == ' '){
+    selectInput("Select a file to process:", "fileSelected");
+  }
+}
+
+void fileSelected(File selection) {
+  if (selection == null) {
+    println("Window was closed or the user hit cancel.");
+  } else {
+    println("selected " + selection.getAbsolutePath());
+    styleImage = loadImage(selection.getAbsolutePath());
+    // resize image (adjust as needed)
+    styleImage.resize(600,400);
+  }
+}
+
+void sendFrameToRunway(){
+  // skip if style image isn't loaded yet
+  if(styleImage == null){
+    return;
+  }
+  // nothing to send if there's no new camera data available
+  if(camera.available() == false){
+    return;
+  }
+  // read a new frame
+  camera.read();
+  // crop image to Runway input format (600x400)
+  PImage image = camera.get(0,0,600,400);
+  // prepare input JSON data to send to Runway
+  JSONObject input = new JSONObject();
+  // set style image
+  input.setString("style_image",ModelUtils.toBase64(styleImage));
+  // set content image
+  input.setString("content_image",ModelUtils.toBase64(image));
+  // set preserve colour
+  input.setBoolean("preserve_color",true);
+  // set alpha
+  input.setFloat("alpha",1.0);
+  // query Runway
+  runway.query(input.toString());
+}
+
+
+// this is called when new Runway data is available
+void runwayDataEvent(JSONObject runwayData){
+  // point the sketch data to the Runway incoming data 
+  String base64ImageString = runwayData.getString("image");
+  // try to decode the image from
+  try{
+    runwayResult = ModelUtils.fromBase64(base64ImageString);
+  }catch(Exception e){
+    e.printStackTrace();
+  }
+  status = "received runway result";
+}
+
+// this is called each time Processing connects to Runway
+// Runway sends information about the current model
+public void runwayInfoEvent(JSONObject info){
+  println(info);
+}
+// if anything goes wrong
+public void runwayErrorEvent(String message){
+  println(message);
+}