Learning Ruby for Force.com Developers – Part 2

This is part #2 from my adventures of learning Ruby. If you missed part #1 you might want to take a look at it just to get up to speed. Again, these are my goals for this series:

• Learn Ruby
• Develop an app locally using Ruby on Rails and the default SQLite database
• Develop an app locally using Ruby on Rails and the SQLite database
• Modify the app to use Database.com and the Force.com Toolkit for Ruby instead of the SQLite database
• Deploy the app to Heroku

I’ve spent the last week or so digging into the Ruby language and have found Ruby Essentials, Ruby Programming and Ruby Tutorial to be especially useful. I’m not an expert, but I do have a feel for it. I’m not going to write a Ruby tutorial (see the links above for that) but I will try to incorporate as much as possible to demonstrate things that I found useful, different or cool.

Write an App in Ruby

The best way to really learn a language is to actually write an app. So that’s what I’m going to do. I expect that my approach and/or assumptions may need to be tweaked once I get further down the road but we’ll see. I’m going to rewrite my Dreamforce 2010 VMforce app in Ruby and Rails. In a nutshell this is a shopping cart app with a little twist. I do a lot of work for Medisend International, which is a non-profit that ships medical supplies to developing countries (among other things). They have an (old) international aid self-service portal that allows aid recipients (typically hospital administrators or local NGOs) to create a shipment and select medical supplies to be shipped to their country. If all goes well I may build the app out and put it into production on Heroku.

So let’s get started by looking at the domain objects for the app. We’ll need a Shipment object (of course) to manage our multiple shipments and an InventoryItem object which will represent the medical items that will be added to the shipment. These are modeled as Custom Object in my development org.

Before we start building out the objects, let’s step back and take a look at how Ruby works with objects. Apex isn’t exactly Java but it’s somewhat similar and a lot of Force.com developers come from a Java background. So it might help to compare Ruby and Java.

As with Java, you’ll find the following similar to Ruby:

• Memory is managed for you via a garbage collector.
• Objects are strongly typed.
• There are public, private, and protected methods.
• There are embedded doc tools (Ruby’s is called RDoc). The docs generated by rdoc look very similar to those generated by javadoc.

Unlike Java, you’ll find the following different in Ruby:

• You don’t need to compile your code. You just run it directly.
• There are different GUI toolkits. Ruby users can try WxRuby, FXRuby, Ruby-GNOME2, or the bundled-in Ruby Tk for example.
• You use the end keyword after defining things like classes, instead of having to put braces around blocks of code.
• You have require instead of import.
• All member variables are private. From the outside, you access everything via methods.
• Parentheses in method calls are usually optional and often omitted.
• Everything is an object, including numbers like 2 and 3.14159.
• There’s no static type checking.
• Variable names are just labels. They don’t have a type associated with them.
• There are no type declarations. You just assign to new variable names as-needed and they just “spring up” (i.e. a = [1,2,3] rather than int[] a = {1,2,3};).
• There’s no casting. Just call the methods. Your unit tests should tell you before you even run the code if you’re going to see an exception.
• It’s foo = Foo.new( “hi”) instead of Foo foo = new Foo( “hi” ).
• The constructor is always named “initialize” instead of the name of the class.
• You have “mixin’s” instead of interfaces.
• YAML tends to be favored over XML.
• It’s nil instead of null.
• == and equals() are handled differently in Ruby. Use == when you want to test equivalence in Ruby (equals() is Java). Use equal?() when you want to know if two objects are the same (== in Java).

Shipment Class

The basic Shipment class looks like the following. It has two instance variables (name and country) and a getter and setter for each one.

class Shipment
  def name
    @name
  end
 
  def name=(value)
    @name = value
  end
 
  def country
    @country
  end
 
  def country=(value)
    @country = value
  end
 
  def initialize()
    puts 'Hello Shipment!!'
  end
end

The following script creates a new instance of the Shipment class, populates the instance variables and display them.

require './Shipment' # similar to Java import
s = Shipment.new # create a new instance
s.name = 'ANG0903-COH1'
s.country = 'Angola'
puts s.name
puts s.country

Create a new directory called ‘RubyDemo’ (or whatever you’d like), switch to that new directory and create a file called Shipment.rb. Paste the contents of the Shipment class above into it. Create another file in that new directory called testshipment.rb and paste the test code above into it. Now open Terminal and switch to this new directory and run ruby testshipment.rb. You should see the following:

So now that we have our Shipment object we need to create our InventoryItem objects to add it. Here’s the class for our InventoryItem:

# base class for all items
class InventoryItem
  attr_accessor :name, :itemNumber, :category, :status, :type
  def initialize(name, itemNumber, category, type)
    @name, @itemNumber, @category, @type = name, itemNumber, category, type
    @status = 'Available'
  end
 
  # override the 'to string' method
  def to_s
    puts 'Item: ' + itemNumber + ', Name: ' + name + ', Category: ' + category
  end
end

You’ll notice that this class is a little different than the Shipment class. Instead of manually creating getter and setter methods, I used “attr_accessor” followed by the names of all of the instance variables for my class. With this command, Ruby will generate  basic getters and setters automatically for me. I could have used “attr_reader” to generate only the getter methods and “attr_writer” to generate the setters.

The initialize method is a standard Ruby class method and is the method which gets called first after an object based on this class has completed initialization. Four arguments are passed into this method and they are used to set instance variable for the object. You’ll also notice that @status is set to ‘Available’ by default each time a new object is created. The to_s method overrides the standard to_s (“to string”) method and displays some information about the item.

Ruby Inheritance

Unfortunately Medisend doesn’t ship just “inventory items”; they ship supplies, equipment and biomedical items. Each one of these types of items is similar but also slightly different. All items have a name, unique item number, category and status however supplies also have expiration dates and quantities of items in the box while equipment has a weight attribute. So instead of making classes for each type of InventoryItem with the same attributes we can use InventoryItem as a base class for each type of item and inherit the shared attributes and functionality from the this base class. (BTW… in salesforce.com there is a Custom Object called InventoryItem__c and Supply, Equipment and Biomed are the types of recordtypes.)

So here’s the SupplyItem that extends the InventoryItem class. The initialize method has 4 arguments; three of which are used to construct the InventoryItem class via the super() call and one that sets the instance variable @quantity. The method also sets a default expiration date just for fun.

require 'date'
require './InventoryItem'
 
class SupplyItem < InventoryItem
  attr_accessor :quantity, :condition, :expirationDate
  def initialize(name, itemNumber, category, quantity)
    super(name, itemNumber, category, 'Supply')
    @quantity = quantity
    @expirationDate = Date.new(2012, 01, 01)
  end
end

The EquipmentItem class also extends InventoryItem but has it’s own to_s method allowing for a more detailed representation of item.

require './InventoryItem'
 
class EquipmentItem < InventoryItem
  attr_accessor :weight
  def initialize(name, itemNumber, category, weight)
    super(name, itemNumber, category, 'Equipment')
    @weight = weight
  end
 
  # provide a custom 'to string' method for equipment only
  def to_s
    puts 'Item: ' + itemNumber + ', Name: ' + name + ', Category: ' + category + ', Weight: ' + weight.to_s + ' lbs'
  end
end

The BiomedItem class is very similar to the SupplyItem class but adds its own weight and condition instance variables.

require './InventoryItem'
 
class BiomedItem < InventoryItem
  attr_accessor :weight, :condition
  def initialize(name, itemNumber, category, weight)
    super(name, itemNumber, category, 'Biomed')
    @weight = weight
    @condition = 'Unknown'
  end
end

Shipment Functionality

With all of the inventory items complete we need to turn our attention back to the Shipment class. The Shipment class should maintain a collection of InventoryItems that are assigned to it and also provide public methods to add and remove items. It should also give some kind of display of the shipment contents. Take a look at the code below along with the comments.

class Shipment
  # class (static) variable 
  @@totalShipmentItems = 0
  # constant
  MAX_ITEMS = 10
  # provides getters and setters for instance variables
  attr_accessor :name, :country, :type, :status, :items
 
  # init the object & set instance variables
  def initialize(name, country, type)
    @name, @country, @type = name, country, type
    @status = 'Not Shipped'
    @items = Hash.new
  end
 
  # adds an item to the Hash of shipment items
  def addItem(item)
    items[item.itemNumber] = item # item number is the key
    @@totalShipmentItems += 1 # increment total items
  end
 
  # deletes an item from the Hash of shipment items
  def deleteItem(itemNumber)
    items.delete(itemNumber) # deleted by key
    @@totalShipmentItems -= 1 # decrements total items
  end
 
  # displays item number and name of each item in shipment
  def displayItems()
    puts 'These are the items in the shipment:'
    items.each {|key, value| puts " #{key} -- #{value.name}" }
  end
 
  # displays item number of each item in shipment
  def displayItemNumbers()
    puts 'These are the item numbers in the shipment:'
    puts items.keys
  end
 
  # displays number of items for this plus all shipments
  def numberOfItems()
    puts 'Number of items in the shipment: ' + items.length.to_s
    puts 'Total items for all shipments: ' + @@totalShipmentItems.to_s
    puts 'Max items: ' + MAX_ITEMS.to_s
  end
end

Now for the fun part. Let’s write a script that utilizes all of our new classes. The following script will do the following:

• Prompt the user to type in the name of a new shipment
• Create and display a new supply item
• Create and display a new equipment item
• Create and display a new biomed item
• Create a new shipment with the name that the user entered
• Add the supply, equipment and biomed items to the shipment
• Display the shipment’s instance members as well as information about the items
• Remove the equipment item
• Display the information about the items

Save the following code in a new file called test.rb.

require './Shipment'
require './SupplyItem'
require './EquipmentItem'
require './BiomedItem'
 
puts "Enter a new shipment name: "  
STDOUT.flush  
shipmentName = gets.chomp  
 
s = SupplyItem.new('Surgical Gloves - Size 7', 'LP10001', 'Gloves', 500)
puts '=== Sample Supply Item =='
puts 'Name: ' + s.name
puts 'Item #: ' + s.itemNumber
puts 'Category: ' + s.category
puts 'Status: ' + s.status
puts 'Quantity: ' + s.quantity.to_s
puts 'Expiration Date: ' + s.expirationDate.to_s
puts s.to_s
 
e = EquipmentItem.new('Crutches', 'LP50879', 'Mobility', 2.5)
puts '=== Sample Equipment Item =='
puts 'Name: ' + e.name
puts 'Item #: ' + e.itemNumber
puts 'Category: ' + e.category
puts 'Status: ' + e.status
puts 'Weight: ' + e.weight.to_s
puts e.to_s
 
b = BiomedItem.new('Clinical Laboratory - Thermostat; w/ stirrer', 'LP25473', 'Clinical Laboratory', 15)
puts '=== Sample Biomed Item =='
puts 'Name: ' + b.name
puts 'Item #: ' + b.itemNumber
puts 'Category: ' + b.category
puts 'Status: ' + b.status
puts 'Weight: ' + b.weight.to_s
puts 'Condition: ' + b.condition
puts b.to_s
 
ship = Shipment.new(shipmentName,'Albania','40ft Container')
# add items to the shipment
ship.addItem(s)
ship.addItem(e)
ship.addItem(b)
puts '=== Sample Shipment =='
puts 'Name: ' + ship.name
puts 'Country: ' + ship.country
puts 'Type: ' + ship.type
puts 'Status: ' + ship.status
ship.numberOfItems()
ship.displayItems()
ship.displayItemNumbers()
puts '=== Deleting an Inventory Item =='
ship.deleteItem(e.itemNumber)
ship.numberOfItems()
ship.displayItems()
ship.displayItemNumbers()

Open Terminal again and type ruby tests.rb and you should see the following:

OK, so that’s a good overview of what we’ll be building. The next step is getting Rails up and running and beginning work on the web aspect. Any and all comments are welcome!

Loading Data to Salesforce.com with the Bulk Data API using Talend

Ward Loving

Talend has recently acquired the capability of inserting data using the new Salesforce.com Bulk Data API.  This capability gives us two important features:  we can load large volumes of data to Salesforce.com much more quickly than we can using the Standard API, and second, we get feedback directly from Salesforce.com about whether the records were successfully inserted into the system.   Loading data to Salesforce.com with the Bulk API and Talend is a two-step process.  The first step is to prepare the data and create the load file.  The second step is to use the tSalesforceBulkExec component to load the data and get the results.    To demonstrate, the data preparation part of the process, I’ve created a pipe-delimited data file as the sample data we’ll load for this article.  

Name|Phone|Type

Acme Lighting|(800) 555-1212| Prospect

American Information Systems|(222) 222-2222|Customer - Direct

Able Moving Company|(999) 999-9999|Customer - Direct 

The data preparation part of this process will convert this pipe-delimited file to a comma-delimited file --a csv file -- which can be loaded to Salesforce.com.  The tSalesforceBulkExec component can only process files in csv format.  Data in any other form or format will have to be converted to this before it can be loaded using the Bulk API.

To follow along with this example you’ll need to create a new job.  I’ve called mine AccountLoad.  Here is my workspace after dragging out the components for step 1:

Figure 1 – Components to Read Data File

Next, click on the tFileInputDelimited_1 component, and set up the parameters for reading the pipe delimited files:

Figure 2 – Parameters for Pipe-Delimited File

Then you’ll need to click the button labeled “…” to setup the schema for the input file.

Figure 3 – Schema for the Input File

You’ll connect the two components by right clicking on the tFileInputDelimited component and selecting Row -> Main and then dropping the little plug icon on the tSalesforceOutputBulk component.  This will push the schema from the file input component to the Bulk Data file component automatically.  The final step for creating your Bulk Data input file is to set the path of the output file for the tSalesforceOutputBulk component.

Figure 4 - Path for Bulk Data Load file

The second part of the AccountLoad job is setting up the tSalesforceBulkExec component to load data to Salesforce.com.  You’ll also want to drag two tFileOutputDelimited components onto the palette to capture your success and error reports.   I’ve labeled one of these files Success and one of these files Errors.  You can do that on the View panel of the component tab.

Figure 5 - Workspace with tSalesforceBulkExec Component

Set up the Bulk file path for the Bulk Exec component to point to the output file generated by tSalesforceOutputBulk component.  In this case, that is "C:/Talend/bulkDataFile.csv".  You’ll also need to complete the parameters for your destination org: Salesforce WebService URL, Username, and Password.  And finally, you’ll need to update the schema to reflect the columns in the data file.  The process here is similar to the setting up the schema for the pipe-delimited input file itself.   The parameters for your tSalesforceBulkExec parameters should look something like the following:

Figure 6 – Parameters for tSalesforceBulkExec Component

After this, you can create the flow relationship between the BulkExec and the tFileOutputDelimited components.  To do this right click on the tSalesforceBulkExec component and select Row -> Main to connect to the Success output file component.  For the flow to the Error file output component you’ll right click the same way but you’ll choose Row -> Reject.    Your workspace after you’ve done this should look like the following:

Figure 7 - Workspace with Flows from Bulk Data Component

Set up the output files paths, header settings, and delimiters for each of the output file components.    If you click the schema button on the Success file output component then you see that the schema of this component has a couple of additional columns: a “salesforce_id” column and a “salesforce_created” column.

Figure 8 - Schema for the Success file output component

The error component has a similar setup except that an “error” column has been added instead of the “salesforce_id” and “salesforce_created” columns.

Figure 9 - Schema for the Error file output component

The final step in the configuration of our job is the connection of the file generation to the data upload part of the job.  This will let Talend know to start the load to Salesforce only when the file creation process has been completed.  To do this, right click on the tSalesforceOutputBulk component that we created earlier and select Trigger -> On Component Ok.  This completes the configuration.

Figure 10 - Workspace for the completed AccountLoad job

Now run the job.   With luck, you’ll see something similar to the following:

Figure 11 – Running the job

Open your success.csv report to validate your data:

Name,Phone,Type,salesforce_id,salesforce_created

"Acme Lighting","(800) 555-1212","Prospect","0015000000Xn1cmAAB","true"

"American Information Systems","(222) 222-2222","Customer - Direct","0015000000Xn1cnAAB","true"

"Able Moving Company","(999) 999-9999","Customer - Direct","0015000000Xn1coAAB","true"

When you’re using the component to load more realistic data volumes you’ll need to keep an eye on the number of Bulk API Batches you have available within a 24 hour period.    A typical organization has 1000 batches per day.  The Bulk API Monitoring Console can be found in your Salesforce instance under Setup->Monitoring->Bulk Data Load Jobs.  You can adjust the number of rows to commit per batch on the Advanced Settings tab of the tSalesforceBulkExec component.

Figure 12 - Bulk API Parameters

The Bulk API components give users capability that wasn’t previously available in Talend – the ability to see whether a record has been successfully inserted into the system and if not what the problem was.  This feature combined with the capability to load data in large quantities makes Talend a legitimate option for production-ready data loads to Salesforce.com.