Professional Documents
Culture Documents
1)
In this Document
Purpose
Scope
Details
REPLICATION SETUP
1. Setup routing correctly first - to make sure traffic from source to destination will go out through the correct
(cluster or not) interface that is tied to the source zpool.
2. Additionally, setup routing correctly on the target system to make sure traffic to the source is routed back to
the correct (cluster or not) interface
3. Setup the replication target
4. Setup the replication action
5. Perform the initial replication update
6. Select the project to replicate, setup an appropriate schedule (such as 'daily') then enable the action.
APPLIES TO:
Sun Storage 7110 Unified Storage System - Version All Versions and later
Sun ZFS Storage 7420 - Version All Versions and later
Oracle ZFS Storage ZS3-4 - Version All Versions and later
Oracle ZFS Storage ZS3-2 - Version All Versions and later
Sun ZFS Storage 7120 - Version All Versions and later
7000 Appliance OS (Fishworks)
PURPOSE
To provide configuration details, suggestions and recommendations when setting up replication.
SCOPE
To provide configuration assistance when setting up replication on the Series 7000 NAS/ZFS Storage Appliance for the
first time.
DETAILS
Sun ZFS Storage Appliances support snapshot-based replication of projects and shares from a source appliance to any
number of target appliances manually, on a schedule, or continuously. The replication includes both data and
metadata. Remote replication (or just "replication") is a general-purpose feature optimized for the following use cases:
Disaster recovery - Replication can be used to mirror an appliance for disaster recovery. In the event of a
disaster that impacts service of the primary appliance (or even an entire datacenter), administrators activate
service at the disaster recovery site, which takes over using the most recently replicated data. When the
primary site has been restored, data changed while the disaster recovery site was in service can be migrated
back to the primary site and normal service restored. Such scenarios are fully testable before such a disaster
occurs.
Data distribution - Replication can be used to distribute data (such as virtual machine images or media) to
remote systems across the world in situations where clients of the target appliance wouldn't ordinarily be able
to reach the source appliance directly, or such a setup would have prohibitively high latency. One example uses
this scheme for local caching to improve latency of read-only data (like documents).
Disk-to-disk backup - Replication can be used as a backup solution for environments in which tape backups
are not feasible. Tape backup might not be feasible, for example, because the available bandwidth is insufficient
or because the latency for recovery is too high.
Data migration - Replication can be used to migrate data and configuration between Sun ZFS Storage
appliances when upgrading hardware or rebalancing storage. Shadow migration can also be used for this
purpose.
The remote replication feature has several important properties:
Snapshot-based. The replication subsystem takes a snapshot as part of each update operation and sends
either the entire project contents up to the snapshot in the case of a full update. In the case of an incremental
update, only the changes since the last replication snapshot for the same action are sent.
Block-level. Each update operation traverses the filesystem at the block level and sends the appropriate
filesystem data and metadata to the target.
Asynchronous. Because replication takes snapshots and then sends them, data is necessarily committed to
stable storage before replication even begins sending it. Continuous replication effectively sends continuous
streams of filesystem changes, but it's still asynchronous with respect to NAS and SAN clients.
Includes metadata. The underlying replication stream serializes both user data and ZFS metadata, including
most properties configured on the Shares screen. These properties can be modified on the target after the first
replication update completes, though not all take effect until the replication connection is severed. For example,
to allow sharing over NFS to a different set of hosts than on the source. See Manging Replication Targets for
details.
Secure. The replication control protocol used among Sun ZFS Storage Appliances is secured with SSL. Data
can optionally be protected with SSL as well. Appliances can only replicate to/from other appliances after an
initial manual authentication process, see Creating and Editing Targets below.
EXPLANATION
Replication
Peer
A Sun Storage 7000 appliance that has been configured as a replication source or target.
Source
An appliance peer containing data to be replicated to another appliance peer (the target).
Individual appliances can act as both a source and a target, but are only one of these in the
context of a particular replication action.
Target
An appliance peer that will receive and store data replicated from another appliance peer (the
source). This term also refers to a configuration object on the appliance that enables it to
replicate to another appliance.
Group
Action
A configuration object on a source appliance specifying a project or share, a target appliance, and
policy options (including how often to send updates, usage of encryption etc.).
Package
The object on the target side that maps to a source action. Each action on a source is associate
with exactly one action from a source. Loss of either object requires you to create the whole
replication again.
Full update
Incremental
update
Continuous
Replication that starts again right away after the last update has been sent . This is NOT a
synchronous update. Think of it as a incremental update that begins right when the previous
incremental update finishes.
Scheduled
A replication that is done on a scheduled basis according to the rules listed in the action.
Manual
data in storage pools owned by A will always work (even from B), but replication of data in storage pools owned
by B will only work when A owns them.
The system may use one of B's addresses as the client for the peer connection. If this happens, replication of
data in storage pools owned by B will always work (even from A), but replication of data in storage pools owned
by A will only work when A owns them.
If you configure the target when A has private network addresses, the system may use one of these addresses
as the client for the peer connection. If this happens, replication of data in storage pools owned by either A or B
will only work when that pool is owned by A.
If you configure a replication target on A, you MUST NOT use that replication target to configure replication
actions on data in storage pools owned by both A and B. This is essentially the same as the first case above:
replicating data from the pools owned by one head will always work, while replicating data from the pools
owned by the other head will only work when the first head is in the OWNER state.
http://docs.oracle.com/cd/E28317_01/html/E38246/shares__projects__replication.html#shares__projects__replication___man
Replication uses ZFS features, this means you need to apply deferred updates after code upgrades are completed.
Replication was designed to run at the project level. This means that it will take a snapshot of ALL the shares in a
project on the source system, even if you are only replication one share. It also means you will have to remove all
these extra snaps after each successful replication is complete.
REPLICATION SETUP
Ideally, you should configure a dedicated (private) network and use dedicated network interfaces for replication traffic.
1. Setup routing correctly first - to make sure traffic from source to destination will go out
through the correct (cluster or not) interface that is tied to the source zpool.
It may be a good idea to add a host-specific route to the target system IP address via the dedicated network
interface:
eg. Adding a route to target IP address '12.34.56.78' via the 'nge3' network interface
NAS_src:configuration services routing> create
NAS_src:configuration services route (uncommitted)> get
family = (unset)
destination = (unset)
mask = (unset)
gateway = (unset)
interface = (unset)
NAS_src:configuration services route (uncommitted)> set family=IPv4
NAS_src:configuration services route (uncommitted)> set destination=12.34.56.78
NAS_src:configuration services route (uncommitted)> set mask=32
NAS_src:configuration services route (uncommitted)> set gateway=12.34.56.254
NAS_src:configuration services route (uncommitted)> set interface=nge3
NAS_src:configuration services route (uncommitted)> commit
nge0
nge0
ibd0
nge1
nge3
static
dynamic
dynamic
inactive
static
2. Additionally, setup routing correctly on the target system to make sure traffic to the source is
NOTE: If replicating to or from a cluster, routing configuration is CRITICAL to be done before starting the
replication setup
http://docs.oracle.com/cd/E28317_01/html/E38246/shares__projects__replication.html#shares__projects__replication___crea
NOTE: When setting up a replication action, it may be better to set the 'hostname' field to a specific IP address - to
ensure that the replication traffic is forced over a specific network interface (and route)
DESCRIPTION
Target
Unique identifier for the replication target system. This property is specified when an
action is initially configured and immutable thereafter.
Pool
Storage pool on the target where this project will be replicated. This property is
specified when an action is initially configured and not shown thereafter.
Enabled
Mode (CLI: continuous) and Whether this action is being replicated continuously or at manual or scheduled
schedule
intervals. See below for details.
Include Snapshots
Whether replication updates include non-replication snapshots. See below for details.
Limit bandwidth
Specifies a maximum speed for this replication update (in terms of data transferred
over the network per second).
Use SSL
Whether to encrypt data on the wire using SSL. Using this feature can have a
significant impact on per-action replication performance.
State
Read-only property describing whether the action is currently idle, sending an update,
or cancelling an update.
Last sync
Read-only property describing the last time an update was successfully sent. This
value may be unknown if the system has not sent a successful update since boot.
Last attempt
Read-only property describing the last time an update was attempted. This value may
be unknown if the system has not attempted to send an update since boot.
Next update
Read-only property describing when the next attempt will be made. This value could
be a date (for a scheduled update), "manual," or "continuous."
long as replication can keep up with data changes, this results in the minimum data lost in the event of a data-loss
disaster on the source system.
Note that continuous replication is still asynchronous (it schedules the "next" replication iteration as soon as the current
one is finished). Sun Storage appliances do not currently support synchronous replication, which does not consider
data committed to stable storage until it's committed to stable storage on both the primary and secondary storage
systems.
http://docs.oracle.com/cd/E28317_01/html/E38246/shares__projects__replication.html#shares__projects__replication___crea
STATUS
idle
NEXT
manual
continuous = false
include_snaps = false
max_bandwidth = unlimited
use_ssl = false
state = idle
state_description = Idle (no update pending)
last_sync = <unknown>
last_try = <unknown>
next_update = manual
6. Select the project to replicate, setup an appropriate schedule (such as 'daily') then enable the
action.
(The frequency can be selected from one of "halfhour", "hour", "day", "week" or "month")
FREQUENCY
day
DAY
-
HH:MM
23:05