<jupyter_start><jupyter_text>VOLTTRON ChargePoint Collector Notebook This notebook sets up a ChargePoint device and forwards datafrom one VOLLTRON instance (this Collector) to another instance (the Aggregator).Most of the notebook's setup and execution is done with shell commands, called from Python.  Setup: Prepare the Volttron Environment VOLTTRON must be installed before using this notebook. For detailed instructions oninstalling and configuring a VOLTTRON/Jupyter server environment, see [Jupyter Notebooks](http://volttron.readthedocs.io/en/devguides/supporting/utilities/JupyterNotebooks.html) in VOLTTRON ReadTheDocs.As is described in that guide, environment variables should have been defined before starting the Jupyter server:````$ export VOLTTRON_ROOT=~/repos/volttron````        (path of the VOLTTRON repository, installed prior to running bootstrap)````$ export VOLTTRON_HOME=~/.volttron````        (directory in which the VOLTTRON instance runs)The first VOLTTRON instance on a server usually runs, by convention, in ~/.volttron.If multiple VOLTTRON instances are to be run on a single host, each must have its own VOLTTRON_HOME.Also before starting the Jupyter server, a VOLTTRON virtual environment should have been activated by executing the following in $VOLTTRON_ROOT:````$ source env/bin/activate````The Python code below does some initialization to prepare for the steps that follow.<jupyter_code>import datetime
import json
import os
import pprint
import sqlite3
import subprocess
import sys
import time

# Define a "run this shell command" method, wrapping subprocess.check_output()
def _sh(shell_command, shell=True, stderr=None):
    try:
        return_value = subprocess.check_output(shell_command, shell=shell, stderr=stderr)
    except Exception, err:
        print('Shell command failed: {}', shell_command)
        print(err)
        return_value = 'Error'
    return return_value

# Same as _sh(), except that this also prints the command output, preceded by an optional label.
def _print_sh(shell_command, label=None, **kwargs):
    print('{0}: {1}\n'.format(label+':' if label else '', _sh(shell_command, **kwargs)))

# Set up local variables vhome and vroot.
# The environment variables VOLTTRON_ROOT and VOLTTRON_HOME should already be defined -- see above.
vroot = %env VOLTTRON_ROOT
vhome = %env VOLTTRON_HOME
print("VOLTTRON_ROOT={}".format(vroot))
print("VOLTTRON_HOME={}".format(vhome))

# Define a VIP_SOCKET environment variable for use while installing and running agents.
socket_name = 'ipc://' + vhome + '/run/vip.socket'
%env VIP_SOCKET=$socket_name

# Run from the VOLTTRON root directory.
os.chdir(vroot)

print("Initialization complete")<jupyter_output><empty_output><jupyter_text>Setup: Shut Down All Agents This ensures a clean agent installation process by the notebook.<jupyter_code>print('Wait for the list to be displayed, and confirm that no agents are listed as running...\n')

# Shut down all agents.
_sh('volttron-ctl shutdown')

# List agent status to verify that the status of each agent is 0 or blank.
_print_sh('volttron-ctl status', stderr=subprocess.STDOUT)<jupyter_output><empty_output><jupyter_text>Setup: Discover the Collector's Network Parameters In order for this Collector to forward data to an Aggregator, the Aggregatormust know the Collector's network parameters, storing them in its known_hosts file.Discover those parameters now.Copy the vip-address's IP and port, and the serverkey,to the Aggregator notebook under 'Setup: Add Each Collector to the known_hosts File',and execute that notebook's code to add this Collector to known_hosts.<jupyter_code># Obtain this server's IP address, volttron port number (usually 22916), and server key:
print('Obtaining network parameters and server key; please wait...\n')
_print_sh('curl ifconfig.me', label='Public IP address')
_print_sh('volttron-ctl auth serverkey', label='Serverkey')
_print_sh('cat {}/config'.format(vhome), label='Config file')<jupyter_output><empty_output><jupyter_text>Setup: Configure the Aggregator's Network Parameters This Collector forwards data to an Aggregator, so it must beconfigured with the Aggregator's IP address, port number and server key.Define those parameters here. Obtain them from the Aggregator notebook,'Setup: Discover the Aggregator's Network Parameters'.<jupyter_code>aggregator_vip_address = '172.16.58.3'
aggregator_vip_port = '22916'
aggregator_server_key = '<KEY>'

aggregator_vip = "tcp://{0}:{1}".format(aggregator_vip_address, aggregator_vip_port)

print('vip = {0}'.format(aggregator_vip))
print('aggregator_server_key = {0}'.format(aggregator_server_key))<jupyter_output><empty_output><jupyter_text>Setup: Test the TCP Connection The ForwardHistorian will send requests to the VOLTTRON Aggregator instancevia TCP commands. Test that the Aggregator instance is capable of receivingTCP requests on the designated IP address and port.If this test fails, the port may not be open on the other server (firewall issue?),the request may be for the wrong IP address and/or port ID,or the other server's VOLTTRON instance may be down or incorrectly configured.<jupyter_code># Use an 'nc' (netcat) command to test the TCP connection
shell_command = 'nc -z -vv -w5 {0} {1}'.format(aggregator_vip_address, aggregator_vip_port)
_print_sh(shell_command, label='Network connection test result', stderr=subprocess.STDOUT)<jupyter_output><empty_output><jupyter_text>Setup: Configure a ForwardHistorian Create a configuration file for this collector's ForwardHistorian.The file specifies the Aggregator's IP address, port and server key,and indicates which topics should be forwarded.<jupyter_code>config = """{{
    "destination-vip": "{0}",
    "destination-serverkey": "{1}",
    "required_target_agents": [],
    "custom_topic_list": [],
    "services_topic_list": ["devices"],
    "topic_replace_list": [
        {{
            "from": "FromString", 
            "to": "ToString"
        }}
    ]
}}""".format(aggregator_vip, aggregator_server_key)
print("config = {}".format(config))
config_path = vhome + '/my_chargepoint_forwarder.config'
with open(config_path, 'w') as file:
    file.write(config)
print('Forwarder configuration written to {}'.format(config_path))<jupyter_output><empty_output><jupyter_text>Configure a ChargePoint Device Driver ** ISSUE: The ChargePoint driver's configuration file needs to include authentication parameters.**** Before running the following, update examples/configurations/drivers/chargepoint1.config with a valid username and password.**<jupyter_code># Change these values for the target ChargePoint device
chargepoint_station_id = '**Put Chargepoint station ID here.**'
chargepoint_username = '**Put Chargepoint username here.**'
chargepoint_password = '**Put <PASSWORD> here.**'

def install_driver_csv(name=None, csv=None):
    _sh('volttron-ctl config store platform.driver {0} {1} --csv'.format(name, csv))

def install_driver_config(name=None, config=None):
    _sh('volttron-ctl config store platform.driver {0} {1}'.format(name, config))

# Create a points file for the ChargePoint device.
points = '''Volttron Point Name,Attribute Name,Register Name,Port #,Type,Units,Starting Value,Writable,Notes
stationID,stationID,StationRegister,,string,Format similar to 1:00001,,FALSE,
stationManufacturer,stationManufacturer,StationRegister,,string,String,,FALSE,
stationModel,stationModel,StationRegister,,string,String,,FALSE,
portNumber,portNumber,StationRegister,1,string,Integer,,FALSE,
stationName,stationName,StationRegister,1,string,String,,FALSE,
stationMacAddr,stationMacAddr,StationRegister,,string,String (colon separated mac address),,FALSE,
stationSerialNum,stationSerialNum,StationRegister,,string,String,,FALSE,
Address,Address,StationRegister,,string,String,,FALSE,
City,City,StationRegister,,string,String,,FALSE,
State,State,StationRegister,,string,String,,FALSE,
Country,Country,StationRegister,,string,String,,FALSE,
postalCode,postalCode,StationRegister,,string,US Postal code,,FALSE,
Lat,Lat,StationRegister,1,float,Latitude Coordinate,,FALSE,
Long,Long,StationRegister,1,float,Longitude Coordinate,,FALSE,
Reservable,Reservable,StationRegister,1,bool,T/F,,FALSE,
Level,Level,StationRegister,1,string,"L1, L2, L3",,FALSE,
Mode,Mode,StationRegister,1,int,"1,2,3",,FALSE,
Voltage,Voltage,StationRegister,1,float,Configured Voltage,,FALSE,
Current,Current,StationRegister,1,float,Configured Current,,FALSE,
Power,Power,StationRegister,1,float,Configured Power,,FALSE,Power supported (kW).
numPorts,numPorts,StationRegister,,int,Integer,,FALSE,Number of Ports
Type,Type,StationRegister,,int,Integer or None,,FALSE,
startTime,startTime,StationRegister,,datetime,Datetime,,FALSE,
endTime,endTime,StationRegister,,datetime,Datetime,,FALSE,
minPrice,minPrice,StationRegister,,float,Dollar Amount,,FALSE,
maxPrice,maxPrice,StationRegister,,float,Dollar Amount,,FALSE,
unitPricePerHour,unitPricePerHour,StationRegister,,float,Dollar Amount,,FALSE,
unitPricePerSession,unitPricePerSession,StationRegister,,float,Dollar Amount,,FALSE,
unitPricePerKWh,unitPricePerKWh,StationRegister,,float,Dollar Amount,,FALSE,
unitPriceForFirst,unitPriceForFirst,StationRegister,,float,Dollar Amount,,FALSE,
unitPricePerHourThereafter,unitPricePerHourThereafter,StationRegister,,float,Dollar Amount,,FALSE,
sessionTime,sessionTime,StationRegister,,datetime,,,FALSE,
Description,Description,StationRegister,1,string,String,,FALSE,
mainPhone,mainPhone,StationRegister,,string,Phone Number,,FALSE,
orgID,orgID,StationRegister,,string,,,FALSE,
organizationName,organizationName,StationRegister,,string,,,FALSE,
sgID,sgID,StationRegister,,string,,,FALSE,
sgName,sgName,StationRegister,,string,,,FALSE,
currencyCode,currencyCode,StationRegister,,string,,,FALSE,
Status,Status,StationStatusRegister,1,string,,,FALSE,"AVAILABLE, INUSE, UNREACHABLE, UNKNOWN "
Status.TimeStamp,TimeStamp,StationStatusRegister,1,datetime,,,FALSE,Timestamp of the last communication between the station and ChargePoint
Connector,Connector,StationRegister,1,string,,,FALSE,"Connector type. For example: NEMA 5-20R, J1772, ALFENL3, "
shedState,shedState,LoadRegister,1,integer,0 or 1,0,TRUE,True when load shed limits are in place
portLoad,portLoad,LoadRegister,1,float,kw,,FALSE,Load in kw
allowedLoad,allowedLoad,LoadRegister,1,float,kw,,TRUE,Allowed load in kw when shedState is True
percentShed,percentShed,LoadRegister,1,integer,percent,,TRUE,Percent of max power shed when shedState is True
alarmType,alarmType,AlarmRegister,,string,,,FALSE,eg. 'GFCI Trip'
alarmTime,alarmTime,AlarmRegister,,datetime,,,FALSE,
clearAlarms,clearAlarms,AlarmRegister,,int,,0,TRUE,Sends the clearAlarms query when set to True
stationRightsProfile,stationRightsProfile,StationRightsRegister,,dictionary,,,FALSE,"Dictionary of sgID, rights name tuples."
sessionID,sessionID,ChargingSessionRegister,1,string,,,FALSE,
startTime,startTime,ChargingSessionRegister,1,datetime,,,FALSE,
endTime,endTime,ChargingSessionRegister,1,datetime,,,FALSE,
Energy,Energy,ChargingSessionRegister,1,float,,,FALSE,
rfidSerialNumber,rfidSerialNumber,ChargingSessionRegister,1,string,,,FALSE,
driverAccountNumber,driverAccountNumber,ChargingSessionRegister,1,string,,,FALSE,
driverName,driverName,ChargingSessionRegister,1,string,,,FALSE,'''
# print "points file contents = {}".format(points)

csv_path = vhome + '/my_chargepoint.csv'
with open(csv_path, 'w') as file:
    file.write(points)
print('ChargePoint points file written to {}\n'.format(csv_path))
    
# Create a config file for the device
config = """{{
    "driver_config": {{
        "stationID" : "{0}",
        "username" : "{1}",
        "password" : <PASSWORD>}",
        "cacheExpiration" : 40
    }},
    "driver_type": "chargepoint",
    "registry_config":"config://{3}",
    "interval": 5,
    "timezone": "US/Pacific",
    "heart_beat_point": "Heartbeat"
}}""".format(chargepoint_station_id, 
             chargepoint_username,
             chargepoint_password,
             'my_chargepoint.csv')
print "config = {}".format(config)

config_path = vhome + '/my_chargepoint.config'
with open(config_path, 'w') as file:
    file.write(config)
print('ChargePoint configuration written to {}'.format(config_path))
    
# Store the configurations in the master driver.
print('\nWait for the platform driver config to display, then confirm that this config appears in it...')
install_driver_csv(name='my_chargepoint.csv', csv=csv_path)
install_driver_config(name='devices/my_chargepoint', config=config_path)

# List the driver configuration to confirm that the drivers were installed successfully.
_print_sh('volttron-ctl config list platform.driver')<jupyter_output><empty_output><jupyter_text>Setup: Install Agents Install each agent employed by the Collector: a master driver, a ForwardHistorian, and 2 Volttron Central agents.<jupyter_code>print('Wait for the list to be displayed, then confirm that all of these agents appear in it...')

def install_agent(dir=None, id=None, config=None, tag=None):
    script_install_command = 'python scripts/install-agent.py -s {0} -i {1} -c {2} -t {3} -f'
    _sh(script_install_command.format(dir, id, config, tag))
    print('Installed {}'.format(tag))

# Install the MasterDriver agent which runs the Bacnet driver
install_agent(dir=vroot+'/services/core/MasterDriverAgent/',
              id='platform.driver',
              config=vroot+'/services/core/MasterDriverAgent/master-driver.agent',
              tag='platform.driver')

# Install a ForwardHistorian agent that forwards metrics to another VOLTTRON instance
install_agent(dir=vroot+'/services/core/ForwardHistorian',
              id='forward_historian',
              config=vhome+'/my_chargepoint_forwarder.config',
              tag='forward_historian')

# Install a Platform Agent
install_agent(dir=vroot+'/services/core/VolttronCentralPlatform',
              id='platform.agent',
              config=vroot+'/services/core/VolttronCentralPlatform/config', 
              tag='vcp')

# Install a Volttron Central Agent
install_agent(dir=vroot+'/services/core/VolttronCentral',
              id='volttron.central',
              config=vroot+'/services/core/VolttronCentral/config', 
              tag='vc')

# List agent status to verify that the agents were installed successfully.
_print_sh('volttron-ctl status', stderr=subprocess.STDOUT)<jupyter_output><empty_output><jupyter_text>Setup: Get the Collector's forward_historian Credentials The Collector's ForwardHistorian agent needs to authenticate to the Aggregator. Authentication is facilitated by adding the agent's credentials to the Aggregator's auth.json file.Copy the PUBLICKEY from the command output below. On the Aggregator, run `volttron-ctl auth add` from the command line. When prompted for credentials, paste the key.<jupyter_code>_print_sh('volttron-ctl auth publickey --tag forward_historian')<jupyter_output><empty_output><jupyter_text>Execution: Refresh Variables and Stop Agents Before starting up the agents, refresh all variables and make sure that all agents are stopped.<jupyter_code>print('Make a fresh start - refresh variable definitions, shut down any running agents, refresh the database')

import datetime
import json
import os
import pprint
import sqlite3
import subprocess
import sys
import time

# Define a "run this shell command" method, wrapping subprocess.check_output()
def _sh(shell_command, shell=True, stderr=None):
    try:
        return_value = subprocess.check_output(shell_command, shell=shell, stderr=stderr)
    except Exception, err:
        print('Shell command failed: {}', shell_command)
        print(err)
        return_value = 'Error'
    return return_value

# Same as _sh(), except that this also prints the command output, preceded by an optional label.
def _print_sh(shell_command, label=None, **kwargs):
    print('{0}: {1}\n'.format(label+':' if label else '', _sh(shell_command, **kwargs)))

# Set up local variables vhome and vroot.
# The environment variables VOLTTRON_ROOT and VOLTTRON_HOME should already be defined -- see above.
vroot = %env VOLTTRON_ROOT
vhome = %env VOLTTRON_HOME
print("VOLTTRON_ROOT={}".format(vroot))
print("VOLTTRON_HOME={}".format(vhome))

# Define a VIP_SOCKET environment variable for use while installing and running agents.
socket_name = 'ipc://' + vhome + '/run/vip.socket'
%env VIP_SOCKET=$socket_name

# Run from the VOLTTRON root directory.
os.chdir(vroot)

# Shut down all agents.
_sh('volttron-ctl shutdown')

# List agent status to verify that the status of each agent is 0 or blank.
_print_sh('volttron-ctl status', stderr=subprocess.STDOUT)<jupyter_output><empty_output><jupyter_text>Execution: Start the agents<jupyter_code>print('Wait for the list to be displayed, then confirm that each started agent is running...')

_sh('volttron-ctl start --tag platform.driver')
_sh('volttron-ctl start --tag forward_historian')
_sh('volttron-ctl start --tag vcp')
_sh('volttron-ctl start --tag vc')

# List agent status to verify that the started agents have status "running".
_print_sh('volttron-ctl status', stderr=subprocess.STDOUT)<jupyter_output><empty_output><jupyter_text>Shutdown: Stop all agents<jupyter_code># Stop all agents.
_sh('volttron-ctl shutdown')

# Verify that all agents have been stopped.
_print_sh('volttron-ctl status', stderr=subprocess.STDOUT)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup