tag:blogger.com,1999:blog-48187044528693176262024-03-19T04:14:00.772-07:00My Technical BlogThe main purpose of this blog is to share my input into Open Source development and my experience, and knowledge gained through 10 years of professional work in ICT industry.Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.comBlogger33125tag:blogger.com,1999:blog-4818704452869317626.post-37378927525451896952019-02-18T04:57:00.000-08:002019-02-18T04:57:21.730-08:00Swift Global Cluster (Multi-Region)<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
<a href="https://docs.openstack.org/swift/latest/overview_global_cluster.html">Swift Global Cluster</a> [1] is a feature that allows Swift to span across multiple regions. By default Swift operates in a single-region mode. Setting up Swift Global Cluster is not difficult, but the configuration overhead is as usual very high. Fortunately there are application modelling tools like <a href="https://jujucharms.com/">Juju</a> [2] available which facilitate software installation and configuration. I have recently added support for Swift Global Cluster feature to <a href="https://jujucharms.com/q/swift">Swift charms</a> [3]. In the following article I will present how to setup Swift in multi-region mode with Juju.</div>
<div style="text-align: justify;">
<br /></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Design</span></h3>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Let's assume that you have two geographically-distributed sites: <i>dc1 </i>and <i>dc2</i>, and you want to deploy Swift regions 1 and 2 in them respectively. We will use Juju for modelling purposes and <a href="https://maas.io/">MaaS</a> [4] as a provider for Juju. Each site has MaaS installed, configured and three nodes enlisted, and commissioned in MaaS. 10.0.1.0/24 and 10.0.2.0/24 subnets are routed and there are no restrictions between them. The whole environment is managed from a Juju client which is external to the sites. The above concept is presented in the following figure:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtsL1sld4QC821Eda4C1vPy7Hvh0dFLjfSbULAV3qO2XOf2chiQUnkakzq0BuLTkejK-qbbM5o9-66-Kn_ohoJAhNyZ3EVKK_tExSpn2T3Y3YHLgWkXvgawcM8xlFk9Gkf4r7xjq154Wzm/s1600/Swift+Global+Cluster.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" data-original-height="405" data-original-width="959" height="268" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjtsL1sld4QC821Eda4C1vPy7Hvh0dFLjfSbULAV3qO2XOf2chiQUnkakzq0BuLTkejK-qbbM5o9-66-Kn_ohoJAhNyZ3EVKK_tExSpn2T3Y3YHLgWkXvgawcM8xlFk9Gkf4r7xjq154Wzm/s640/Swift+Global+Cluster.jpg" title="Swift Global Cluster (Multi-Region)" width="640" /></a></div>
<div>
<br /></div>
<div style="text-align: justify;">
Each node will host Swift storage services and <a href="https://linuxcontainers.org/lxd/introduction/">LXD container</a> [5] with Swift proxy service. Swift proxy will be deployed in the HA mode. Each node belongs to a different zone and has 3 disks: <i>sdb</i>, <i>sdc </i>and <i>sdd </i>for object storing purposes. The end goal is to have 3 replicas of the object in each site.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
P.S.: If you have more than two sites, don't worry. Swift Global Cluster scales out, so can easily add another regions later on.</div>
<div style="text-align: justify;">
<br /></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Initial deployment</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
Let's assume that you already have Juju client installed, two MaaS clouds added to the client and Juju controllers bootstrapped in each cloud. If you don't know how to do it, you can refer to Juju documentation [2]. You can list Juju controllers by executing the following command:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju list-controllers</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Controller Model User Access Cloud/Region Models Machines HA Version</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">juju-dc1* default admin superuser maas-dc1 2 1 none 2.5.1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">juju-dc2 default admin superuser maas-dc2 2 1 none 2.5.1</span></div>
</div>
<div>
<br /></div>
<div style="text-align: justify;">
NOTE: Make sure you use Juju version 2.5.1 or later.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The asterisk character indicates the current controller in use. You can switch between them by executing the following command:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju switch <controller_name></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Before we start we have to download patched charms from the branches I created (they haven't been merged with the upstream code yet):</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ cd /tmp</span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ git clone </span><span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">git@github.com:tytus-kurek/charm-swift-proxy.git</span></span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ git clone </span></span><span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">git@github.com:tytus-kurek/charm-swift-storage.git</span></span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ cd charm-swift-proxy</span></span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ git checkout </span></span><span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">1815879</span></span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ cd ../charm-swift-storage</span></span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ git checkout </span></span><span style="text-align: left;"><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">1815879</span></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Then we create Juju bundles which will be used to deploy the models:</span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span style="color: orange;"><br /></span></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ cat <<EOF > /tmp/swift-dc<span style="color: orange;">1</span>.yaml</span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><div>
series: bionic</div>
<div>
services:</div>
<div>
<div>
swift-storage-dc<span style="color: orange;">1</span>-zone1:</div>
<div>
charm: /tmp/charm-swift-storage</div>
<div>
num_units: 1</div>
<div>
options:</div>
<div>
block-device: sdb sdc sdd</div>
<div>
region: <span style="color: orange;">1</span></div>
<div>
zone: 1</div>
</div>
<div>
<div>
swift-storage-dc<span style="color: orange;">1</span>-zone2:</div>
<div>
charm: /tmp/charm-swift-storage</div>
<div>
num_units: 1</div>
<div>
options:</div>
<div>
block-device: sdb sdc sdd</div>
<div>
region: <span style="color: orange;">1</span></div>
<div>
zone: 2</div>
</div>
<div>
<div>
swift-storage-dc<span style="color: orange;">1</span>-zone3:</div>
<div>
charm: /tmp/charm-swift-storage</div>
<div>
num_units: 1</div>
<div>
options:</div>
<div>
block-device: sdb sdc sdd</div>
<div>
region: <span style="color: orange;">1</span></div>
<div>
zone: 3</div>
</div>
<div>
swift-proxy-dc<span style="color: orange;">1</span>:</div>
<div>
charm: /tmp/charm-swift-proxy</div>
<div>
num_units: 3</div>
<div>
options:</div>
<div>
enable-multi-region: true</div>
<div>
read-affinity: "<span style="color: orange;">r1=100, r2=200</span>"</div>
<div>
region: "<span style="color: orange;">RegionOne</span>"</div>
<div>
replicas: 3</div>
<div>
vip: "10.0.<span style="color: orange;">1</span>.254"</div>
<div>
write-affinity: "<span style="color: orange;">r1, r2</span>"</div>
<div>
write-affinity-node-count: 3</div>
<div>
zone-assignment: manual</div>
<div>
to:</div>
<div>
- lxd:0</div>
<div>
- lxd:1</div>
<div>
- lxd:2</div>
<div>
haproxy-swift-proxy-dc<span style="color: orange;">1</span>:</div>
<div>
charm: cs:haproxy</div>
<div>
relations:</div>
<div>
- [ "haproxy-swift-proxy-dc<span style="color: orange;">1</span>:ha", "swift-proxy-dc<span style="color: orange;">1</span>:ha" ]</div>
<div>
- [ "swift-proxy-dc<span style="color: orange;">1</span>:swift-storage", "swift-storage-dc<span style="color: orange;">1</span>-zone1:swift-storage" ]</div>
<div>
- [ "swift-proxy-dc<span style="color: orange;">1</span>:swift-storage", "swift-storage-dc<span style="color: orange;">1</span>-zone2:swift-storage" ]</div>
<div>
- [ "swift-proxy-dc<span style="color: orange;">1</span>:swift-storage", "swift-storage-dc<span style="color: orange;">1</span>-zone3:swift-storage" ]</div>
</span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">EOF</span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ cat <<EOF > /tmp/swift-dc<span style="color: orange;">2</span>.yaml</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><div>
series: bionic</div>
<div>
services:</div>
<div>
<div>
swift-storage-dc<span style="color: orange;">2</span>-zone1:</div>
<div>
charm: /tmp/charm-swift-storage</div>
<div>
num_units: 1</div>
<div>
options:</div>
<div>
block-device: sdb sdc sdd</div>
<div>
region: <span style="color: orange;">2</span></div>
<div>
zone: 1</div>
</div>
<div>
<div>
swift-storage-dc<span style="color: orange;">2</span>-zone2:</div>
<div>
charm: /tmp/charm-swift-storage</div>
<div>
num_units: 1</div>
<div>
options:</div>
<div>
block-device: sdb sdc sdd</div>
<div>
region: <span style="color: orange;">2</span></div>
<div>
zone: 2</div>
</div>
<div>
<div>
swift-storage-dc<span style="color: orange;">2</span>-zone3:</div>
<div>
charm: /tmp/charm-swift-storage</div>
<div>
num_units: 1</div>
<div>
options:</div>
<div>
block-device: sdb sdc sdd</div>
<div>
region: <span style="color: orange;">2</span></div>
<div>
zone: 3</div>
</div>
<div>
swift-proxy-dc<span style="color: orange;">2</span>:</div>
<div>
charm: /tmp/charm-swift-proxy</div>
<div>
num_units: 3</div>
<div>
options:</div>
<div>
enable-multi-region: true</div>
<div>
read-affinity: "<span style="color: orange;">r2=100, r1=200</span>"</div>
<div>
region: "<span style="color: orange;">RegionTwo</span>"</div>
<div>
replicas: 3</div>
<div>
vip: "10.0.<span style="color: orange;">2</span>.254"</div>
<div>
write-affinity: "<span style="color: orange;">r2, r1</span>"</div>
<div>
write-affinity-node-count: 3</div>
<div>
zone-assignment: manual</div>
<div>
to:</div>
<div>
- lxd:0</div>
<div>
- lxd:1</div>
<div>
- lxd:2</div>
<div>
haproxy-swift-proxy-dc<span style="color: orange;">2</span>:</div>
<div>
charm: cs:haproxy</div>
<div>
relations:</div>
<div>
- [ "haproxy-swift-proxy-dc<span style="color: orange;">2</span>:ha", "swift-proxy-dc<span style="color: orange;">2</span>:ha" ]</div>
<div>
- [ "swift-proxy-dc<span style="color: orange;">2</span>:swift-storage", "swift-storage-dc<span style="color: orange;">2</span>-zone1:swift-storage" ]</div>
<div>
- [ "swift-proxy-dc<span style="color: orange;">2</span>:swift-storage", "swift-storage-dc<span style="color: orange;">2</span>-zone2:swift-storage" ]</div>
<div>
- [ "swift-proxy-dc<span style="color: orange;">2</span>:swift-storage", "swift-storage-dc<span style="color: orange;">2</span>-zone3:swift-storage" ]</div>
</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">EOF</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
Note that we mark all storage nodes in <i>dc1</i> as Swift region 1 and all storage nodes in <i>dc2</i> as Swift region 2. The affinity settings of Swift proxy application will be used to determine how the data will be read and written.</div>
<div>
<br /></div>
<div>
Finally we create the models and deploy the bundles:</div>
</div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span style="color: orange;"><br /></span></span></div>
<div style="text-align: justify;">
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju switch juju-dc<span style="color: orange;">1</span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span style="color: orange;"><br /></span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju add-model swift-dc<span style="color: orange;">1</span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span style="color: orange;"><br /></span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju deploy /tmp/swift-dc<span style="color: orange;">1</span>.yaml</span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju switch juju-dc<span style="color: orange;">2</span></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span style="color: orange;"><br /></span></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju add-model swift-dc<span style="color: orange;">2</span></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span style="color: orange;"><br /></span></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju deploy /tmp/swift-dc<span style="color: orange;">2</span>.yaml</span></div>
</div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">This takes a while. Monitor Juju status and wait until all units in both models enter the <i>active </i>state.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<h3 style="text-align: center;">
<span style="color: #b45f06; font-family: inherit;">Setting up Swift Global Cluster</span></h3>
<div>
<span style="color: #b45f06; font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">In order to setup Swift Global Cluster we have to relate storage nodes from <i>dc1</i> with the Swift proxy application in <i>dc2</i> and <i>vice versa. </i>Moreover a master-slave relation has to be established between <i>swift-proxy-dc1 </i>and <i>swift-proxy-dc2 </i>applications. </span>However, as they don't belong to the same model / controller / cloud, we have to create <a href="https://docs.jujucharms.com/devel/en/models-cmr">offers</a> [6] first (offers allow cross-model / cross-controller / cross-cloud relations creation):</div>
<div style="text-align: justify;">
<span style="font-family: inherit;"> </span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small; text-align: justify;">$ juju switch juju-dc</span><span style="color: orange; font-family: "Courier New", Courier, monospace; font-size: x-small; text-align: justify;">1</span></div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju offer swift-proxy-dc<span style="color: orange;">1</span>:master swift-proxy-dc<span style="color: orange;">1</span>-master</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju offer swift-proxy-dc<span style="color: orange;">1</span>:swift-storage swift-proxy-dc<span style="color: orange;">1</span>-swift-storage</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju switch juju-dc<span style="color: orange;">2</span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju offer swift-proxy-dc<span style="color: orange;">2</span>:swift-storage swift-proxy-dc<span style="color: orange;">2</span>-swift-storage</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: inherit;">Then consume the offers:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju switch juju-dc<span style="color: orange;">1</span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju consume maas-dc<span style="color: orange;">2</span>:admin/swift-proxy-dc<span style="color: orange;">2</span>-swift-storage</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju switch juju-dc<span style="color: orange;">2</span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju consume maas-dc<span style="color: orange;">1</span>:admin/swift-proxy-dc<span style="color: orange;">1</span>-master</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju consume maas-dc<span style="color: orange;">1</span>:admin/swift-proxy-dc<span style="color: orange;">1</span>-swift-storage</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
Add required relations:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju switch juju-dc<span style="color: orange;">1</span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju relate swift-storage-dc<span style="color: orange;">1</span>-zone1 swift-proxy-dc<span style="color: orange;">2</span>-swift-storage</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">$ juju relate swift-storage-dc<span style="color: orange;">1</span>-zone2 swift-proxy-dc<span style="color: orange;">2</span>-swift-storage</span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">$ juju relate swift-storage-dc<span style="color: orange;">1</span>-zone3 swift-proxy-dc<span style="color: orange;">2</span>-swift-storage</span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">$ juju switch juju-dc2</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju relate swift-storage-dc<span style="color: orange;">2</span>-zone1 swift-proxy-dc<span style="color: orange;">1</span>-swift-storage</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">$ juju relate swift-storage-dc<span style="color: orange;">2</span>-zone2 swift-proxy-dc<span style="color: orange;">1</span>-swift-storage</span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">$ juju relate swift-storage-dc<span style="color: orange;">2</span>-zone3 swift-proxy-dc<span style="color: orange;">1</span>-swift-storage</span></div>
</div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">$ juju relate swift-proxy-dc<span style="color: orange;">2</span>:slave swift-proxy-dc<span style="color: orange;">1</span>-master</span></div>
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: inherit;">Finally increase the replication factor to 6:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju switch juju-dc<span style="color: orange;">1</span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju config swift-proxy-dc<span style="color: orange;">1</span> replicas=6</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju switch juju-dc<span style="color: orange;">2</span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju config swift-proxy-dc<span style="color: orange;">2</span> replicas=6</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">This setting together with the affinity settings will cause that in each site 3 replicas of the object will be created.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<h3 style="text-align: center;">
<span style="color: #b45f06; font-family: inherit;">Site failure</span></h3>
<div>
<span style="color: #b45f06; font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
At this point we have Swift Global Cluster configured. There are two sites and each of them is acting as a different Swift region. As each node belongs to a different zone and the replication factor has been set to 6, each storage node is hosting 1 replica of each object. Both proxies can be used to read and write the data. Such cluster is highly available and geo-redundant. This means it can survive a failure of any site, however, due to an eventual consistency nature of Swift, some data can be lost during the failure event.</div>
<div style="text-align: justify;">
<br /></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Failover</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
In case of the <i>dc1</i> failure the Swift Proxy application in <i>dc2 </i>can be used to read and write the data in both regions. However, if <i>dc1</i> cannot be recovered, <i>swift-proxy-dc2 </i>has to be manually transitioned to master, so that another regions could be deployed. In order to transition <i>swift-proxy-dc2 </i>to master execute the following command:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju switch juju-dc2</span></div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">$ juju config swift-proxy-dc2 enable-transition-to-master=True</span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="text-align: center;">Not that this should be used with an extra caution. After that another regions can be deployed based on the instructions from the previous sections. Don't forget to update the affinity settings when deploying additional regions. </span></div>
<div>
<br /></div>
<div>
[1] <a href="https://docs.openstack.org/swift/latest/overview_global_cluster.html">https://docs.openstack.org/swift/latest/overview_global_cluster.html</a></div>
<div>
<br /></div>
<div>
[2] <a href="https://jujucharms.com/">https://jujucharms.com/</a></div>
<div>
<br /></div>
<div>
[3] <a href="https://jujucharms.com/q/swift">https://jujucharms.com/q/swift</a></div>
<div>
<br /></div>
<div>
[4] <a href="https://maas.io/">https://maas.io/</a></div>
<div>
<br /></div>
<div>
[5] <a href="https://linuxcontainers.org/lxd/introduction/">https://linuxcontainers.org/lxd/introduction/</a></div>
<div>
<br /></div>
<div>
[6] <a href="https://docs.jujucharms.com/devel/en/models-cmr">https://docs.jujucharms.com/devel/en/models-cmr</a></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com2tag:blogger.com,1999:blog-4818704452869317626.post-12430998680839911242019-01-24T05:25:00.004-08:002020-02-16T12:53:48.865-08:00How to make Samsung Xpress C480W scanner working on Ubuntu Bionic<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
As I don't have time today, this is going to be one of the shortest posts in this blog. But I really want to save it as I've already spent a couple of hours trying to figure it out. So to make the long story short I reinstalled my laptop with Ubuntu Bionic and my Samsung Xpress C480W scanner stopped working. Nooooo!</div>
<div>
<br /></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Fixing the scanner</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
OK, so in order to make the scanner working again download and install the Samsung Unified Linux Driver:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">sudo apt install libusb-0.1-4</span></div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">cd /tmp</span></div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">wget https://www.bchemnet.com/suldr/driver/UnifiedLinuxDriver-1.00.39.tar.gz</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">tar -xzf </span><span style="font-family: "courier new" , "courier" , monospace;">UnifiedLinuxDriver-1.00.39.tar.gz</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">cd uld</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">./install.sh</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Easy peasy. So what's wrong? The problem is that the installer places the module under the "/usr/lib/sane" directory while Ubuntu Bionic expects them under the "/usr/lib/x86_64-linux-gnu/sane" directory! Sigh ... I don't know whether this is a bug or not, I basically didn't have time to check. But the problem can be easily solved by linking the module into the proper location:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">sudo ln -sf /usr/lib/sane/libsane-smfp.so* /usr/lib/x86_64-linux-gnu/sane/</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">I hope it helps.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-43950530876404517942018-10-03T08:11:00.001-07:002018-10-04T02:33:07.900-07:00How to setup Canonical Identity Service (Candid) in the HA mode<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Canonical Identity Service (Candid) does not require any HA-specific extensions / settings and can be deployed in the HA mode in various ways. Which one is the best one, however? It is hard to say, but the reference architecture is always the most obvious choice. In the following post we will setup Candid in the HA mode based on the following tool set:</div>
<div>
<ul style="text-align: left;">
<li style="text-align: justify;"><b>PostgreSQL</b> - reliable and highly available backend</li>
<li style="text-align: justify;"><b>Corosync & Pacemaker</b> - messaging and service management</li>
<li style="text-align: justify;"><b>HAProxy</b> - load balancing and SSL termination</li>
</ul>
</div>
<div style="text-align: justify;">
This architecture is shown in the following diagram:</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR46kGitLHuy0378JmmbQjaVjdu5g9Dgf48DUxEnpMBRu36BJv2jvG0X0SX0e8hoB9WffArpaDKYs3cjle9n6p1t1n5A3SX2pgK-QwcQJBWCgRHU4zb_NYyk4qZAz6LjYiqyICzVzjlqeS/s1600/Candid+HA+Reference+Architecture.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" data-original-height="375" data-original-width="632" height="378" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR46kGitLHuy0378JmmbQjaVjdu5g9Dgf48DUxEnpMBRu36BJv2jvG0X0SX0e8hoB9WffArpaDKYs3cjle9n6p1t1n5A3SX2pgK-QwcQJBWCgRHU4zb_NYyk4qZAz6LjYiqyICzVzjlqeS/s640/Candid+HA+Reference+Architecture.jpg" title="Canonical Identity Service (Candid) HA (High Availability)" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The following part contains step-by-step instructions to install and configure Candid in the HA mode. All services are set on LXD containers with Ubuntu Bionic. It is assumed that LDAP is used as the Identity Provider.</div>
<div style="text-align: justify;">
<br /></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Prerequisites</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<ul style="text-align: left;">
<li style="text-align: justify;"><a href="https://help.ubuntu.com/lts/serverguide/certificates-and-security.html.en">Generate CA-signed certificates</a> for the following services:</li>
<ul>
<li style="text-align: justify;">Candid - <i>candid.crt</i> + <i>candid.key </i>with FQDN of <i>candid.example.com</i></li>
<li style="text-align: justify;">PostgreSQL - <i>server.crt + server.key</i> with FQDN of <i>postgres.example.com</i></li>
</ul>
<li style="text-align: justify;"><a href="https://www.ubuntu.com/containers/lxd">Launch LXD containers</a>:</li>
<ul>
<li style="text-align: justify;"><i>candid-ha-0</i></li>
<li style="text-align: justify;"><i>candid-ha-1</i></li>
<li style="text-align: justify;"><i>candid-ha-2</i></li>
</ul>
<li style="text-align: justify;">Ensure the following FQDNs are resolvable (e.g. by editing <i style="text-align: justify;">/etc/hosts</i><span style="text-align: justify;"> </span><span style="text-align: justify;">file on containers):</span></li>
<ul>
<li style="text-align: justify;"><i>candid.example.com</i> (pointing to Candid VIP)</li>
<li style="text-align: justify;"><i>ldap.example.com</i> (pointing to Identity Provider IP)</li>
<li style="text-align: justify;"><i>postgres.example.com</i> (pointing to PostgreSQL VIP)</li>
</ul>
<li><a href="http://manpages.ubuntu.com/manpages/bionic/man8/update-ca-certificates.8.html" style="text-align: justify;">Ensure CA certificate (<i>/etc/ssl/certs/ca.pem</i>) is installed on all containers</a></li>
</ul>
<div>
<br /></div>
</div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">PostgreSQL</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<div>
On all containers install and stop PostgreSQL:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># apt -y install postgresql</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># systemctl stop postgresql</span></div>
<div>
<br /></div>
<div>
On <i>candid-ha-0</i> container disable PostgreSQL:</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># systemctl disable postgresql</span></div>
<div>
<br /></div>
<div>
On all containers change ownership and permissions of PostgreSQL certificate and key:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># chown postgres:postgres /var/lib/postgresql/10/main/server.crt</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># chown postgres:postgres /var/lib/postgresql/10/main/server.key</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># chmod 600 /var/lib/postgresql/10/main/server.key</span></div>
<div>
<br /></div>
<div>
On all containers configure PostgreSQL:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># sed -i "s^ssl_cert_file = '/etc/ssl/certs/ssl-cert-snakeoil.pem'^#ssl_cert_file = '/etc/ssl/certs/ssl-cert-snakeoil.pem'^" /etc/postgresql/10/main/postgresql.conf</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># sed -i "s^ssl_key_file = '/etc/ssl/private/ssl-cert-snakeoil.key'^#ssl_key_file = '/etc/ssl/private/ssl-cert-snakeoil.key'^" /etc/postgresql/10/main/postgresql.conf</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># cat <<EOF >> /etc/postgresql/10/main/postgresql.conf</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">listen_addresses = '*'</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">max_connections = 300</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">wal_level = hot_standby</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">synchronous_commit = on</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">archive_mode = off</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">max_wal_senders = 10</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">wal_keep_segments = 256</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">hot_standby = on</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">restart_after_crash = off</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">hot_standby_feedback = on</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">ssl_cert_file = '/var/lib/postgresql/10/main/server.crt'</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">ssl_key_file = '/var/lib/postgresql/10/main/server.crt'</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">EOF</span></div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># cat /etc/postgresql/10/main/pg_hba.conf | head -n -3 > /tmp/pg_hba.conf; mv /tmp/pg_hba.conf /etc/postgresql/10/main/pg_hba.conf</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># cat <<EOF >> /etc/postgresql/10/main/pg_hba.conf</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">host replication postgres 10.130.194.10/32 trust</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">host replication postgres 10.130.194.11/32 trust</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">host replication postgres 10.130.194.12/32 trust</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">host replication postgres 10.130.194.253/32 trust</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">host replication postgres 10.130.194.254/32 trust</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">hostssl candid candid 10.130.194.10/32 md5</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">hostssl candid candid 10.130.194.11/32 md5</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">hostssl candid candid 10.130.194.12/32 md5</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">hostssl candid candid 10.130.194.253/32 md5</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">hostssl candid candid 10.130.194.254/32 md5</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">EOF</span></div>
<div>
<br /></div>
<div>
<b>NOTE</b>: Replace IP addresses with IP addresses of all containers and IP addresses reserved for Candid and PostgreSQL VIP.</div>
<div>
<br /></div>
<div>
On <i>candid-ha-1</i> container start PostgreSQL:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># systemctl start postgresql</span></div>
<div>
<br /></div>
<div>
On <span style="font-family: inherit;"><i>candid-ha-2</i> </span>container initiate the replication and start PostgreSQL:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># rm -rf /var/lib/postgresql/10/main/</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># sudo -u postgres pg_basebackup -h 10.130.194.11 -D /var/lib/postgresql/10/main -v --wal-method=stream</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># systemctl start postgresql</span></div>
<div>
<br /></div>
<div>
<b>NOTE</b>: Replace IP addresses with IP addresses of <i>candid-ha-1</i> container.</div>
<div>
<br /></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">HAProxy</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<div>
On all containers install HAProxy:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># apt -y install haproxy</span></div>
<div>
<br /></div>
<div>
On all containers configure HAProxy:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">cat <<EOF > /etc/haproxy/haproxy.cfg</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">defaults</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> timeout connect 10s</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> timeout client 1m</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> timeout server 1m</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">global</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> tune.ssl.default-dh-param 2048</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">frontend candid</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> bind *:443 ssl crt /etc/ssl/private/candid.pem</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> reqadd X-Forwarded-Proto:\ https</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> option http-server-close</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> default_backend candid</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">backend candid</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> balance source</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> hash-type consistent</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> server candid-ha-1 10.130.194.10:8081 check</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> server candid-ha-2 10.130.194.11:8081 check</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> server candid-ha-3 10.130.194.12:8081 check</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">EOF</span></div>
<div>
<br /></div>
<div>
<b>NOTE: </b><i>candid.pem</i> is a concatenation of <i>candid.crt</i> and <i>candid.key</i> files (<span style="font-family: Courier New, Courier, monospace;">cat candid.crt candid.key > candid.pem</span>)<br />
<b><br /></b>
<b>NOTE</b>: Replace IP addresses with IP addresses of containers.</div>
<div>
<br /></div>
<div>
On all containers restart HAProxy:</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># systemctl restart haproxy</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<h3 style="text-align: center;">
<span style="color: #b45f06; font-family: inherit;">Corosync & Pacemaker</span></h3>
<div>
<span style="color: #b45f06; font-family: inherit;"><br /></span></div>
<div>
<div>
On all containers install Corosync and Pacemaker:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># apt -y install crmsh</span></div>
<div>
<br /></div>
<div>
On all containers configure Corosync:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># cat <<EOF > /etc/corosync/corosync.conf</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">totem {</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> version: 2</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> token: 3000</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> token_retransmits_before_loss_const: 10</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> join: 60</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> consensus: 3600</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> vsftype: none</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> max_messages: 20</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> clear_node_high_bit: yes</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> secauth: off</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> threads: 0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> ip_version: ipv4</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> rrp_mode: none</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> transport: udpu</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">}</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">quorum {</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> provider: corosync_votequorum</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> }</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">nodelist {</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> node {</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> ring0_addr: 10.130.194.10</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> nodeid: 1000</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> }</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> node {</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> ring0_addr: 10.130.194.11</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> nodeid: 1001</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> }</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> node {</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> ring0_addr: 10.130.194.12</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> nodeid: 1002</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> }</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">}</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">logging {</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> fileline: off</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> to_stderr: yes</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> to_logfile: no</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> to_syslog: yes</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> syslog_facility: daemon</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> debug: off</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> logger_subsys {</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> subsys: QUORUM</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> debug: off</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> }</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">}</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">EOF</span></div>
<div>
<br /></div>
<div>
<b>NOTE</b>: Replace IP addresses with IP addresses of containers.</div>
<div>
<br /></div>
<div>
On all containers restart Corosync:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># systemctl restart corosync</span></div>
<div>
<br /></div>
<div>
On <i>candid-ha-0</i> container configure Pacemaker (run the command and replace the data):</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># crm configure edit</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">node 1000: candid-ha-0 \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> attributes pgsql-data-status=DISCONNECT</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">node 1001: candid-ha-1 \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> attributes pgsql-data-status=LATEST</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">node 1002: candid-ha-2 \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> attributes pgsql-data-status=DISCONNECT</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">primitive haproxy lsb:haproxy \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> op monitor interval=15s</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">primitive pgsql pgsql \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> params rep_mode=sync pgctl="/usr/lib/postgresql/10/bin/pg_ctl" psql="/usr/bin/psql" pgdata="/var/lib/postgresql/10/main/" socketdir="/var/run/postgresql" config="/etc/postgresql/10/main/postgresql.conf" logfile="/var/log/postgresql/postgresql-10-ha.log" master_ip=10.130.194.253 node_list="candid-ha-0 candid-ha-1 candid-ha-2" primary_conninfo_opt="keepalives_idle=60 keepalives_interval=5 keepalives_count=5" restart_on_promote=true \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> op start timeout=60s interval=0s on-fail=restart \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> op monitor timeout=60s interval=4s on-fail=restart \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> op monitor timeout=60s interval=3s on-fail=restart role=Master \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> op promote timeout=60s interval=0s on-fail=restart \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> op demote timeout=60s interval=0s on-fail=stop \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> op stop timeout=60s interval=0s on-fail=block \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> op notify timeout=60s interval=0s</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">primitive res_candid_vip IPaddr2 \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> params ip=10.130.194.254 cidr_netmask=32 \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> op monitor interval=10s \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> meta migration-threshold=0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">primitive res_pgsql_vip IPaddr2 \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> params ip=10.130.194.253 cidr_netmask=32 \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> op monitor interval=10s \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> meta migration-threshold=0 target-role=Started</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">ms ms_pgsql pgsql \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">clone haproxy-clone haproxy</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">location cli-prefer-res_pgsql_vip res_pgsql_vip role=Started inf: candid-ha-0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">order ord_demote 0: ms_pgsql:demote res_pgsql_vip:stop symmetrical=false</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">order ord_promote inf: ms_pgsql:promote res_pgsql_vip:start symmetrical=false</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">location pgsql_on_two_nodes ms_pgsql -inf: candid-ha-0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">colocation pgsql_vip inf: res_pgsql_vip ms_pgsql:Master</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">property cib-bootstrap-options: \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> have-watchdog=false \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> dc-version=1.1.18-2b07d5c5a9 \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> cluster-infrastructure=corosync \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> cluster-name=debian \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> stonith-enabled=false \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> last-lrm-refresh=1534598484</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">rsc_defaults rsc-options: \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> resource-stickiness=INFINITY \</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> migration-threshold=10</span></div>
<div style="color: #b45f06; font-family: inherit;">
<br /></div>
</div>
<div>
<div style="font-family: inherit;">
<b>NOTE</b>: Replace IP addresses with IP addresses reserved for Candid and PostgreSQL VIP.</div>
<div style="font-family: inherit;">
<br /></div>
<h3 style="font-family: inherit; text-align: center;">
<span style="color: #b45f06;">Candid</span></h3>
<div style="font-family: inherit;">
<span style="color: #b45f06;"><br /></span></div>
<div>
<div>
On all containers install Candid:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># apt -y install candid</span></div>
<div>
<br /></div>
<div>
On candid-ha-1 container create PostgreSQL user and database for Candid:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># su postgres -c "createuser candid -P"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># su postgres -c "createdb candid -O candid"</span></div>
<div>
<br /></div>
<div>
On all containers export <i>CANDID_URL</i> variable:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># export CANDID_URL="https://candid.example.com"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># echo "export CANDID_URL=\"https://candid.example.com\"" >> /root/.bashrc</span></div>
<div>
<br /></div>
<div>
On <i>candid-ha-0 </i>container create admin credentials for Candid API:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># candid put-agent --admin --agent-file admin.agent</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># candid put-agent --admin --agent-file services.keys</span></div>
<div>
<br /></div>
<div>
On all containers configure Candid:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># cat <<EOF > /etc/candid/config.yaml</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">listen-address: 10.130.194.10:8081</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">location: 'https://candid.example.com'</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">private-addr: 10.130.194.10</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">storage:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> type: postgres</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> connection-string: dbname=candid user=candid password=candid host=postgres.example.com</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">private-key: Xh/hbA92cqSAFunu3IgVK0VeZrZvtpR7E50OXR39S48=</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">public-key: Olf//8WpzSnIFm0HwJX4WCoXlTkw1ndAAvFGP1nj71U=</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">admin-agent-public-key: JsJOh7kXuONBGvgF2kunmbn+gcg8MpoBfMVMrB8RrTw=</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">resource-path: /usr/share/candid/</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">access-log: /var/log/identity/access.log</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">identity-providers:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - type: ldap</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> name: ldap</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> domain: example</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> url: ldap://ldap.example.com/dc=example,dc=com</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> ca-cert: |</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> -----BEGIN CERTIFICATE-----</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> MIIEGzCCAwOgAwIBAgIJAI67J2tCUZWMMA0GCSqGSIb3DQEBCwUAMIGjMQswCQYD</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> VQQGEwJQTDETMBEGA1UECAwKU29tZS1TdGF0ZTEPMA0GA1UEBwwGS3Jha293MRww</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> GgYDVQQKDBNJVHN0ZWVyIFR5dHVzIEt1cmVrMRMwEQYDVQQLDApDb25zdWx0aW5n</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> MRgwFgYDVQQDDA93d3cuaXRzdGVlci5jb20xITAfBgkqhkiG9w0BCQEWEm9mZmlj</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> ZUBpdHN0ZWVyLmNvbTAeFw0xODAyMDExMTQ1NTZaFw0yODAxMzAxMTQ1NTZaMIGj</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> MQswCQYDVQQGEwJQTDETMBEGA1UECAwKU29tZS1TdGF0ZTEPMA0GA1UEBwwGS3Jh</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> a293MRwwGgYDVQQKDBNJVHN0ZWVyIFR5dHVzIEt1cmVrMRMwEQYDVQQLDApDb25z</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> dWx0aW5nMRgwFgYDVQQDDA93d3cuaXRzdGVlci5jb20xITAfBgkqhkiG9w0BCQEW</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> Em9mZmljZUBpdHN0ZWVyLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> ggEBAM3cE1zSJgQw3XNzOn0Z7pcwlHg6B2/ubOQ1L6UDmQNFqdz0Zmg5nSTPpeU6</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> VlxrUz8YogiISEl549v92TjBSw7SrDTexUNqKNeHdF6wdVQpEsU8hZbndP1sgYH8</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> 2ONYTKG1sqs03JS8gdbb8ZBJYQGiqT2owOLU43QTlVl1KE5yq5b7PwgUlqCfSMbG</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> FUlE51YBcbv0DYDILJ5trbslAT3xXCk9Lbxyi7cW87fB9mfvkmd48jZb1yl2EY1V</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> qFiHTrLw0TK+JcI49psxccOy1aXzKJjVbjTt3l/d2mCUIh76S5AlBOiLmn9zOo2G</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> GA0LtTm7lgJVD1kahpf5NbNAVTUCAwEAAaNQME4wHQYDVR0OBBYEFN8zFHBCIM4T</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> BPEBeRonPyf7h7K4MB8GA1UdIwQYMBaAFN8zFHBCIM4TBPEBeRonPyf7h7K4MAwG</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> A1UdEwQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAAvpzYUcMT2Z7ohbUpV94ZOz</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> 8bL919UozNY0V9lrcbnI5v/GlibnNDd/lE7/kBZAdpJMFpzYLxQdBdukXNsQ66fu</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> UCj7OVZbnhykN6aiAmB7NHHb4gp6Eu9Aan5Cfky2UE66FmZRMulNMH+l0B64AJ9h</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> crRUIGpsK0BrLl6KITE7OB9Qbjm8VSsRBxDy1MrdwGjDyeWCVIU5YRGcs/j5X45k</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> OeK8S1KwpuU8/wjkP5lYKUeNRDXIbduWsNAYbLLY8N1wWh+373IuZg3OkfSkIEV7</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> ApcRh/uwdJKsx0ebO8aHTDCiBi4AYGDcAumsmpY1CAaWBDzdja77bQocI3qDV60=</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> -----END CERTIFICATE-----</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> dn: cn=admin,dc=example,dc=com</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> password: admin</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> user-query-filter: (objectClass=posixAccount)</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> user-query-attrs:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> id: uid</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> email: mail</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> display-name: displayName</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> group-query-filter: (&(objectClass=groupOfNames)(member={{.User}}))</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">EOF</span></div>
<div>
<br /></div>
<div>
<b>NOTE</b>: Replace the following entries:</div>
<div>
<ul style="text-align: left;">
<li><i>listen-address</i> - IP address of the container with port</li>
<li><i>private-addr</i> - IP address of the container</li>
<li><i>password</i> - password of PostgreSQL <i>candid</i> user</li>
<li><i>private-key</i> - value of private from <i>services.keys</i> file</li>
<li><i>public-key</i> - value of public from <i>services.keys</i> file</li>
<li><i>admin-agent-public-key</i> - value of public from <i>admin.keys</i> file</li>
<li><i>domain </i>- LDAP domain</li>
<li><i>url </i>- LDAP URL</li>
<li><i>ca-cert</i> - content of the <i>ca.pem</i> file</li>
<li><i>dn </i>- LDAP bind credentials</li>
<li><i>password </i>- LDAP bind password</li>
</ul>
</div>
<div>
On all containers restart Candid service:</div>
<div>
<br /></div>
<div>
<i># systemctl restart candid</i></div>
<div>
<br /></div>
<div>
At this point you should be able to access Candid at <a href="https://candid.example.com/">https://candid.example.com</a></div>
</div>
</div>
</div>
</div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-40650521022832106802018-06-20T02:59:00.000-07:002018-06-27T02:14:45.049-07:00Circular asynchronous MySQL replication between multiple geographically-distributed Percona XtraDB Clusters with Juju<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<br /></div>
<div style="text-align: justify;">
I have recently shown you how to replicate databases between two Percona XtraDB Clusters using asynchronous MySQL replication with Juju [<a href="http://tkurek.blogspot.com/2018/06/how-to-replicate-databases-between-two_11.html">1</a>]. Today I am going to take you one step further. I will show you how to configure circular asynchronous MySQL replication between geographically-distributed Percona XtraDB Clusters. I will use Juju for this purpose again as it not only simplifies the deployment, but the entire life cycle management. So ... grab a cup of coffee and see the world around you changing!</div>
<div style="text-align: justify;">
<br /></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Design</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
Let's assume that you have three geographically-distributed sites: <i>dc1</i>, <i>dc2</i> and <i>dc3</i>, and you want to replicate <i>example </i>database across Percona XtraDB Clusters located in each site. We will use Juju for modelling purposes and MaaS [<a href="https://docs.maas.io/devel/en/">2</a>] as a provider for Juju. Each site has MaaS installed, configured and nodes enlisted, and commissioned in MaaS. The whole environment is managed from a Juju client which is external to the sites. The above is presented on the following diagram:</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLc4dMHc5JvCqKuvKv4NFIA5y5DN2vu9Vyt2T6jj_799A5m_gPvyYxIdbcdzk75WKQLpyhtM8_f7cH6to7RuxW2eB1F7Tqkj5KoHWfFCiyBRSHUV4TS_u8qtyyjNGQpgbIsDqDzwfW4Wiq/s1600/Percona.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="Percona XtraDB Cluster circular asynchronous MySQL replication" border="0" data-original-height="720" data-original-width="960" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLc4dMHc5JvCqKuvKv4NFIA5y5DN2vu9Vyt2T6jj_799A5m_gPvyYxIdbcdzk75WKQLpyhtM8_f7cH6to7RuxW2eB1F7Tqkj5KoHWfFCiyBRSHUV4TS_u8qtyyjNGQpgbIsDqDzwfW4Wiq/s640/Percona.jpg" title="Percona XtraDB Cluster circular asynchronous MySQL replication" width="640" /></a></div>
<div>
<span style="color: #b45f06;"><br /></span>
<span style="color: #b45f06;">P.S.: If you have more than three sites, don't worry. Circular replication scales out, so can replicate the database across multiple Percona XtraDB Clusters.</span><br />
<span style="color: #b45f06;"><br /></span></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Initial deployment</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
Let's assume that you already have Juju client installed, all your three MaaS clouds added to the client and Juju controllers bootstrapped in each cloud. If you don't know how to do it, you can refer to MaaS documentation [<a href="https://docs.jujucharms.com/devel/en/clouds-maas">3</a>]. You can list Juju controllers by executing the following command:</div>
<div style="text-align: justify;">
<b style="font-family: "Courier New", Courier, monospace;"><span style="font-size: x-small;"><br /></span></b></div>
<div style="text-align: justify;">
<b style="font-family: "Courier New", Courier, monospace;"><span style="font-size: x-small;">$ juju list-controllers</span></b></div>
<div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Controller Model User Access Cloud/Region Models Machines HA Version</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">juju-dc1* default admin superuser maas-dc1 2 1 none 2.3.7</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">juju-dc2 default admin superuser maas-dc2 2 1 none 2.3.7</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">juju-dc3 default admin superuser maas-dc3 2 1 none 2.3.7</span></div>
</div>
</div>
<div style="text-align: justify;">
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
The asterisk character indicates the current controller in use. You can switch between them by executing the following command:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch <controller_name></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
In each cloud, on each controller we create a model and deploy Percona XtraDB Cluster within this model. I'm going to use bundles [<a href="https://docs.jujucharms.com/devel/en/charms-bundles">4</a>] today to make the deployment easier:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">1</span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><span style="color: orange;"><br /></span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju add-model pxc-rep<span style="color: orange;">1</span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><span style="color: orange;"><br /></span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ cat <<EOF > pxc<span style="color: orange;">1</span>.yaml</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"></span><br />
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">series: xenial</span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">
</span>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">services:</span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">
<div>
pxc<span style="color: orange;">1</span>:</div>
<div>
charm: "/tmp/charm-percona-cluster"</div>
<div>
num_units: 3</div>
<div>
options:</div>
<div>
cluster-id: <span style="color: orange;">1</span></div>
<div>
databases-to-replicate: "example"</div>
<div>
root-password: "root"</div>
<div>
vip: 10.0.<span style="color: orange;">1</span>.100</div>
<div>
hacluster-pxc<span style="color: orange;">1</span>:</div>
<div>
charm: "cs:hacluster"</div>
<div>
options:</div>
<div>
cluster_count: 3</div>
<div>
relations:</div>
<div>
- [ pxc<span style="color: orange;">1</span>, hacluster-pxc<span style="color: orange;">1</span> ]</div>
</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">EOF</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju deploy pxc<span style="color: orange;">1</span>.yaml</b></span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">2</span></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><span style="color: orange;"><br /></span></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju add-model pxc-rep<span style="color: orange;">2</span></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><span style="color: orange;"><br /></span></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ cat <<EOF > pxc<span style="color: orange;">2</span>.yaml</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"></span><br />
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">series: xenial</span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">
</span>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">services:</span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">
<div>
pxc<span style="color: orange;">2</span>:</div>
<div>
charm: "/tmp/charm-percona-cluster"</div>
<div>
num_units: 3</div>
<div>
options:</div>
<div>
cluster-id: <span style="color: orange;">2</span></div>
<div>
databases-to-replicate: "example"</div>
<div>
root-password: "root"</div>
<div>
vip: 10.0.<span style="color: orange;">2</span>.100</div>
<div>
hacluster-pxc<span style="color: orange;">2</span>:</div>
<div>
charm: "cs:hacluster"</div>
<div>
options:</div>
<div>
cluster_count: 3</div>
<div>
relations:</div>
<div>
- [ pxc<span style="color: orange;">2</span>, hacluster-pxc<span style="color: orange;">2</span> ]</div>
</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">EOF</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju deploy pxc<span style="color: orange;">2</span>.yaml</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">3</span></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><span style="color: orange;"><br /></span></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju add-model pxc-rep<span style="color: orange;">3</span></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><span style="color: orange;"><br /></span></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ cat <<EOF > pxc<span style="color: orange;">3</span>.yaml</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"></span><br />
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">series: xenial</span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">
</span>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">services:</span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">
<div>
pxc<span style="color: orange;">3</span>:</div>
<div>
charm: "/tmp/charm-percona-cluster"</div>
<div>
num_units: 3</div>
<div>
options:</div>
<div>
cluster-id: <span style="color: orange;">3</span></div>
<div>
databases-to-replicate: "example"</div>
<div>
root-password: "root"</div>
<div>
vip: 10.0.<span style="color: orange;">3</span>.100</div>
<div>
hacluster-pxc<span style="color: orange;">3</span>:</div>
<div>
charm: "cs:hacluster"</div>
<div>
options:</div>
<div>
cluster_count: 3</div>
<div>
relations:</div>
<div>
- [ pxc<span style="color: orange;">3</span>, hacluster-pxc<span style="color: orange;">3</span> ]</div>
</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">EOF</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju deploy pxc<span style="color: orange;">3</span>.yaml</b></span></div>
</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div>
Re-fill your cup of coffee and after some time check the Juju status:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">1</span></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju status</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Model Controller Cloud/Region Version SLA</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc-rep1 juju-dc1 maas-dc1 2.3.7 unsupported</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">App Version Status Scale Charm Store Rev OS Notes</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">hacluster-pxc1 active 3 hacluster local 0 ubuntu </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1 5.6.37-26.21 active 3 percona-cluster local 45 ubuntu </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Unit Workload Agent Machine Public address Ports Message</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1/0* active idle 0 10.0.1.1 3306/tcp Unit is ready</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> hacluster-pxc1/0* active idle 10.0.1.1 Unit is ready and clustered</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1/1 active idle 1 10.0.1.2 3306/tcp Unit is ready</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> hacluster-pxc1/1 active idle 10.0.1.2 Unit is ready and clustered</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1/2 active idle 2 10.0.1.3 3306/tcp Unit is ready</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> hacluster-pxc1/2 active idle 10.0.1.3 Unit is ready and clustered</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Machine State DNS Inst id Series AZ Message</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">0 started 10.0.1.1 juju-83da9e-0 xenial Running</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">1 started 10.0.1.2 juju-83da9e-1 xenial Running</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">2 started 10.0.1.3 juju-83da9e-2 xenial Running</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Relation provider Requirer Interface Type Message</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">hacluster-pxc1:ha pxc1:ha hacluster subordinate </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">hacluster-pxc1:hanode hacluster-pxc1:hanode hacluster peer </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1:cluster pxc1:cluster percona-cluster peer</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div>
<span style="font-family: inherit;">If all units turned to the <i>active </i>state, you're ready to go. Remember to check the status in all models.</span></div>
</div>
<div>
<span style="font-family: inherit;"><br /></span></div>
</div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Setting up circular asynchronous MySQL replication</span></h3>
<div style="text-align: justify;">
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
In order to set up circular asynchronous MySQL replication between all three Percona XtraDB Clusters we have to relate them. However, as they don't belong to the same model / controller / cloud, we have to create offers [<a href="https://docs.jujucharms.com/devel/en/models-cmr">5</a>] first (offers allow cross-model / cross-controller / cross-cloud relations):</div>
<div style="text-align: justify;">
<span style="font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">1</span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju offer pxc<span style="color: orange;">1</span>:slave</b></span></div>
<div style="text-align: justify;">
<span style="font-size: x-small;"><br /></span></div>
<div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">2</span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju offer pxc<span style="color: orange;">2</span>:slave</b></span></div>
</div>
<div>
<div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;"><span style="font-size: x-small;"><br /></span></span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"></span><br />
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">3</span></b></span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">
</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju offer pxc<span style="color: orange;">3</span>:slave</b></span></div>
</div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
Then we have to consume the offers:</div>
<div style="text-align: justify;">
<span style="font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">1</span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju consume juju-dc<span style="color: orange;">2</span>:admin/pxc-rep<span style="color: orange;">2</span>.pxc<span style="color: orange;">2</span> pxc<span style="color: orange;">2</span></b></span></div>
<div>
<div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;"><span style="font-size: x-small;"><br /></span></span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"></span><br />
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">2</span></b></span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">
</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju consume juju-dc<span style="color: orange;">3</span>:admin/pxc-rep<span style="color: orange;">3</span>.pxc<span style="color: orange;">3</span> pxc<span style="color: orange;">3</span></b></span></div>
</div>
<div>
<div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;"><span style="font-size: x-small;"><br /></span></span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"></span><br />
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">3</span></b></span></div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">
</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju consume juju-dc<span style="color: orange;">1</span>:admin/pxc-rep<span style="color: orange;">1</span>.pxc<span style="color: orange;">1</span> pxc<span style="color: orange;">1</span></b></span></div>
</div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><span style="color: orange;"><br /></span></b></span></div>
<div style="text-align: justify;">
Finally, we can add the cross-cloud relations:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">1</span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju relate pxc<span style="color: orange;">1</span>:master pxc<span style="color: orange;">2</span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">2</span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju relate pxc<span style="color: orange;">2</span>:master pxc<span style="color: orange;">3</span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc<span style="color: orange;">3</span></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju relate pxc<span style="color: orange;">3</span>:master pxc<span style="color: orange;">1</span></b></span></div>
</div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><span style="color: orange;"><br /></span></b></span></div>
<div style="text-align: justify;">
Wait a couple of minutes and check whether all units turned into <i>active </i>state:</div>
<div style="text-align: justify;">
<br /></div>
<div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc1</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju status</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Model Controller Cloud/Region Version SLA</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc-rep1 juju-dc1 maas-dc1 2.3.7 unsupported</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">SAAS Status Store URL</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc2 active maas-dc2 admin/pxc-rep2.pxc2</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">App Version Status Scale Charm Store Rev OS Notes</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">hacluster-pxc1 active 3 hacluster local 0 ubuntu </span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1 5.6.37-26.21 active 3 percona-cluster local 45 ubuntu </span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Unit Workload Agent Machine Public address Ports Message</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1/0* active idle 0 10.0.1.1 3306/tcp Unit is ready</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> hacluster-pxc1/0* active idle 10.0.1.1 Unit is ready and clustered</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1/1 active idle 1 10.0.1.2 3306/tcp Unit is ready</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> hacluster-pxc1/1 active idle 10.0.1.2 Unit is ready and clustered</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1/2 active idle 2 10.0.1.3 3306/tcp Unit is ready</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> hacluster-pxc1/2 active idle 10.0.1.3 Unit is ready and clustered</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Machine State DNS Inst id Series AZ Message</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">0 started 10.0.1.1 juju-83da9e-0 xenial Running</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">1 started 10.0.1.2 juju-83da9e-1 xenial Running</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">2 started 10.0.1.3 juju-83da9e-2 xenial Running</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Offer Application Charm Rev Connected Endpoint Interface Role</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1 pxc1 percona-cluster 48 1/1 slave mysql-async-replication requirer</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Relation provider Requirer Interface Type Message</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">hacluster-pxc1:ha pxc1:ha hacluster subordinate </span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">hacluster-pxc1:hanode hacluster-pxc1:hanode hacluster peer </span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1:cluster pxc1:cluster percona-cluster peer</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">pxc1:master pxc2:slave mysql-async-replication regular</span></div>
</div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><br /></span></div>
<div style="text-align: justify;">
At this point you should have circular asynchronous MySQL replication working between all three Percona XtraDB Clusters. The asterisk character in the output above indicates the leader unit. Let's check whether it's actually working by connecting to the MySQL console on the leader unit:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju ssh pxc1/0</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ mysql -u root -p</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">First check whether as a master it has granted access to <i>pxc2 </i>application units:</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>mysql> SELECT Host FROM mysql.user WHERE User='replication';</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">+----------+</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">| Host |</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">+----------+</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">| 10.0.2.1 |</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">| 10.0.2.2 |</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">| 10.0.2.3 |</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">+----------+</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">3 rows in set (0.00 sec)</span></div>
<div style="font-family: inherit;">
<br /></div>
<div style="font-family: inherit;">
Then check its slave status as a slave of <i>pxc3</i>:</div>
<div style="font-family: inherit;">
<b><br /></b></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>mysql> SHOW SLAVE STATUS\G;</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">*************************** 1. row ***************************</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Slave_IO_State: Waiting for master to send event</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_Host: 10.0.3.100</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_User: replication</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_Port: 3306</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Connect_Retry: 60</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_Log_File: mysql-bin.000004</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Read_Master_Log_Pos: 338</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Relay_Log_File: mysqld-relay-bin.000002</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Relay_Log_Pos: 283</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Relay_Master_Log_File: mysql-bin.000004</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Slave_IO_Running: Yes</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Slave_SQL_Running: Yes</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Replicate_Do_DB: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Replicate_Ignore_DB: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Replicate_Do_Table: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Replicate_Ignore_Table: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Replicate_Wild_Do_Table: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Replicate_Wild_Ignore_Table: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Last_Errno: 0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Last_Error: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Skip_Counter: 0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Exec_Master_Log_Pos: 338</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Relay_Log_Space: 457</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Until_Condition: None</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Until_Log_File: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Until_Log_Pos: 0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_SSL_Allowed: No</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_SSL_CA_File: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_SSL_CA_Path: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_SSL_Cert: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_SSL_Cipher: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_SSL_Key: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Seconds_Behind_Master: 0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Master_SSL_Verify_Server_Cert: No</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Last_IO_Errno: 0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Last_IO_Error: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Last_SQL_Errno: 0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Last_SQL_Error: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Replicate_Ignore_Server_Ids: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_Server_Id: 1</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_UUID: e803b085-739f-11e8-8f7e-00163e391eab</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_Info_File: /var/lib/percona-xtradb-cluster/master.info</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> SQL_Delay: 0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> SQL_Remaining_Delay: NULL</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_Retry_Count: 86400</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_Bind: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Last_IO_Error_Timestamp: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Last_SQL_Error_Timestamp: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_SSL_Crl: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Master_SSL_Crlpath: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Retrieved_Gtid_Set: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Executed_Gtid_Set: </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> Auto_Position: 0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">1 row in set (0.00 sec)</span></div>
<div style="font-family: inherit;">
<br /></div>
<div style="font-family: inherit;">
Finally, create a database:</div>
<div style="font-family: inherit;">
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>mysql> CREATE DATABASE example;</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">Query OK, 1 row affected (0.01 sec)</span></div>
<div style="font-family: inherit;">
<br /></div>
<div style="font-family: inherit;">
and check whether it has been created on <i>pxc2</i>:</div>
<div style="font-family: inherit;">
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ juju switch juju-dc2</b></span></div>
<div>
<div style="font-family: inherit;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br class="Apple-interchange-newline" />$ juju ssh pxc2/0</b></span></div>
<div style="font-family: inherit;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div style="font-family: inherit;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>$ mysql -u root -p</b></span></div>
<div style="font-family: inherit;">
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b><br /></b></span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"><b>mysql> show databases;</b></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">+--------------------+</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">| Database |</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">+--------------------+</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">| information_schema |</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">| example |</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">| mysql |</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">| performance_schema |</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">| test |</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">+--------------------+</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">5 rows in set (0.00 sec)</span></div>
</div>
<div style="font-family: inherit;">
<br /></div>
<div style="font-family: inherit;">
It is there! It should be created on <i>pxc3 </i>as well. Go there and check it.</div>
<div style="font-family: inherit;">
<br /></div>
<div style="font-family: inherit;">
At this point you can write to <i>example</i> database from all units of all Percona XtraDB Clusters. This is how circular asynchronous MySQL replication works. Isn't that easy? Of course it is - thanks to Juju!</div>
<div style="font-family: inherit;">
<br /></div>
</div>
</div>
</div>
</div>
</div>
<div style="text-align: center;">
<h3>
<span style="color: #b45f06;">References</span></h3>
</div>
<div>
<br /></div>
<div>
[1] <a href="http://tkurek.blogspot.com/2018/06/how-to-replicate-databases-between-two_11.html">http://tkurek.blogspot.com/2018/06/how-to-replicate-databases-between-two_11.html</a></div>
<div>
<br /></div>
<div>
[2] <a href="https://docs.maas.io/2.1/en/">https://docs.maas.io/devel/en/</a></div>
<div>
<br /></div>
<div>
[3] <a href="https://docs.jujucharms.com/devel/en/clouds-maas">https://docs.jujucharms.com/devel/en/clouds-maas</a></div>
<div>
<br /></div>
<div>
[4] <a href="https://docs.jujucharms.com/devel/en/charms-bundles">https://docs.jujucharms.com/devel/en/charms-bundles</a></div>
<div>
<br /></div>
<div>
[5] <a href="https://docs.jujucharms.com/devel/en/models-cmr">https://docs.jujucharms.com/devel/en/models-cmr</a></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-37366709748177871802018-06-12T08:43:00.002-07:002018-06-12T08:43:18.555-07:00Pacemaker - resource with configured location is not transferred back to the preferred location after recovery from a failure<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: center;">
<span style="color: #b45f06;"> Intro</span></h3>
<div style="text-align: justify;">
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
As in the title - I configured a Pacemaker resource and used the location directive to configure its preferred location. The resource got allocated to the preferred node, but it was not transferred there back after a reboot. It took me a few hours to find the root cause and fix it.</div>
<div>
<br /></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Symptoms</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
Configure a preferred location of Pacemaker resource or resource group:</div>
<div style="text-align: justify;">
<b><br /></b></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><b># crm configure location <location_name> <resource_name> <priority>: <node_name></b></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">For example:</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><b># crm configure location VIP_location VIP 50: node01</b></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
should ensure that the <i>VIP </i>resource is always placed on <i>node01</i>, unless other locations with higher priority for this resource are created.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This works fine until the failure of <i>node01</i>. If it happens the resource is transferred to another node in the cluster, which is expected. However, once <i>node01 </i>recovers from the failure, the resource should be transferred back to its preferred location. The problem is that it is not!</div>
<div style="text-align: justify;">
<br /></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Solution</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
The solution is trivial, but not very well documented. Check the configured value of general cluster resource stickiness. It it's higher or equal to the priority of the location, the resource won't be transferred back to its preferred location after recovery from the failure. So in our example it is enough to execute:</div>
<div>
<br /></div>
<div>
<div>
<b><span style="font-family: Courier New, Courier, monospace;"># crm configure rsc_defaults resource-stickiness=49</span></b></div>
</div>
<div>
<b><span style="font-family: Courier New, Courier, monospace;"><br /></span></b></div>
<div>
That's it! A few hours I said ... Yet I managed to find a time to write this post, however :).</div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-88342011395080946802018-06-11T07:53:00.001-07:002018-06-27T02:12:14.207-07:00How to replicate databases between two Percona XtraDB Clusters - asynchronous MySQL replication with Juju<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<br />
<div style="text-align: justify;">
I have recently been looking for a solution for Percona XtraDB Cluster databases replication across multiple geographically-distributed sites. My first approach wast to use Galera synchronous replication, but I have found that the official recommendation is to use MySQL asynchronous replication [<a href="https://www.percona.com/resources/mysql-white-papers/high-level-multi-datacenter-mysql-high-availability">1</a>]. I have followed one of many available tutorials for setting up replication between two MySQL nodes, but I have quickly faced some issues. Well, setting up replication between one Percona cluster and the other one is not the same as setting up replication between one MySQL node and the other one. Therefore, I have decided to document all of those traps, so that you could avoid them during your deployment. Moreover, I have created a patch for the percona-cluster charm [<a href="https://github.com/tytus-kurek/charm-percona-cluster/tree/1776171">2</a>], so that you could fully automate your multi-site Percona deployment!</div>
<h3 style="text-align: left;">
<div style="text-align: center;">
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: center;">
<span style="color: #b45f06;">Deployment - manual</span></div>
</h3>
<div style="text-align: center;">
<br /></div>
<div style="text-align: justify;">
Assuming you have two Percona clusters (<i>pxc1 </i>and <i>pxc2</i>) already deployed, the following is a list of steps to follow in order to setup master-slave replication from <i>pxc1 </i>to <i>pxc2</i>:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b><span style="color: orange;">1)</span></b> Update <i>my.cnf </i>file on each unit in each cluster to contain the following settings:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: left;">
</div>
<ul style="text-align: left;">
<li style="text-align: justify;"><i><b>pxc1:</b></i></li>
</ul>
<div style="text-align: justify;">
<b><i><br /></i></b></div>
<span style="font-family: "courier new" , "courier" , monospace;">[mysqld]</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">bind-address<span style="white-space: pre;"> </span>= 0.0.0.0</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">log_bin<span style="white-space: pre;"> </span>= /var/log/mysql/mysql-bin.log</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">expire_logs_days<span style="white-space: pre;"> </span>= 10</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">max_binlog_size<span style="white-space: pre;"> </span>= 100M</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">log_slave_updates<span style="white-space: pre;"> </span>= 1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">server_id<span style="white-space: pre;"> </span>= <b>1</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">binlog_do_db<span style="white-space: pre;"> </span>= example1</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">binlog_do_db<span style="white-space: pre;"> </span>= example2</span><br />
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div style="text-align: left;">
</div>
<ul style="text-align: left;">
<li style="text-align: justify;"><i><b>pxc2:</b></i></li>
</ul>
<div style="text-align: justify;">
<div style="text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"><br class="Apple-interchange-newline" />[mysqld]</span></div>
<div style="text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">bind-address<span style="white-space: pre;"> </span>= 0.0.0.0</span></div>
<div style="text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">log_bin<span style="white-space: pre;"> </span>= /var/log/mysql/mysql-bin.log</span></div>
<div style="text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">expire_logs_days<span style="white-space: pre;"> </span>= 10</span></div>
<div style="text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">max_binlog_size<span style="white-space: pre;"> </span>= 100M</span></div>
<div style="text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">log_slave_updates<span style="white-space: pre;"> </span>= 1</span></div>
<div style="text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">server_id<span style="white-space: pre;"> </span>= <b>2</b></span></div>
<div style="text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">binlog_do_db<span style="white-space: pre;"> </span>= example1</span></div>
<div style="text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">binlog_do_db<span style="white-space: pre;"> </span>= example2</span></div>
</div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Note that the value of the <i>server_id </i>setting must be different for each cluster, but it must be the same within the cluster [<a href="https://www.percona.com/forums/questions-discussions/percona-xtradb-cluster/9433-master-master-between-two-clusters?p=19622#post19622">3</a>]. Also note that you have to create separate <i>binlog_do_db </i>entry for each database to replicate.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Once done restart <i>mysql </i>service:</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b># systemctl restart mysql</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="color: orange; font-family: inherit; font-weight: bold;">2) </span><span style="font-family: inherit;">Create <i>replication </i>user account on <i>pxc1 </i>cluster side by executing the following commands from any unit:</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<b><span style="font-family: "courier new" , "courier" , monospace;">$ mysql -u root -p</span></b></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>mysql> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'</b></span><b style="font-family: "Courier New", Courier, monospace; text-align: justify;">10.130.194.21</b><b style="font-family: "Courier New", Courier, monospace;">' IDENTIFIED BY 'password';</b></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>mysql> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'</b></span><b style="font-family: "Courier New", Courier, monospace; text-align: justify;">10.130.194.22</b><b style="font-family: "Courier New", Courier, monospace;">' IDENTIFIED BY 'password';</b></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>mysql> GRANT REPLICATION SLAVE ON *.* TO 'replication'@'</b></span><b style="font-family: "Courier New", Courier, monospace; text-align: justify;">10.130.194.23</b><b style="font-family: "Courier New", Courier, monospace;">' IDENTIFIED BY 'password';</b></div>
<div style="text-align: justify;">
<b style="font-family: "Courier New", Courier, monospace;"><br /></b></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Note that you have to execute the <i>GRANT </i>command for each of the <i>pxc2 </i>cluster members.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Finally, check master status and note down the output as it will be used in point 3:</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>mysql></b> <b>SHOW MASTER STATUS;</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">+------------------+----------+-------------------+------------------+</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">+------------------+----------+-------------------+------------------+</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">| mysql-bin.000001 | 618 | example1,example2 | |</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">+------------------+----------+-------------------+------------------+</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">1 row in set (0.00 sec)</span></div>
</div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><span style="font-family: inherit;"><b><span style="color: orange;">3)</span> </b>Setup replication on <i>pxc2 </i>cluster side by executing the following commands from any unit:</span></span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;">$ mysql -u root -p</span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;">mysql> STOP SLAVE;</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;">mysql> </span><span style="font-family: "courier new" , "courier" , monospace;"><b>CHANGE MASTER TO master_host='10.130.194.254', master_port=3306, master_user='replication', master_password='password', master_log_file='mysql-bin.000001', master_log_pos=618;</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;">mysql> START SLAVE;</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">Note that you have to use the file and position from point 2.</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><b><span style="color: orange;">4)</span></b> At this point you have master-slave replication working from <i>pxc1 </i>to <i>pxc2</i>. You can check it by logging to any unit of <i>pxc1</i> and executing the following commands:</span></div>
<div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;">mysql> CREATE DATASE example1;</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace; font-weight: 700;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">The database is now visible on all units of both <i>pxc1 </i>and <i>pxc2 </i>clusters:</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><b><br /></b></span></div>
<div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>mysql> SHOW DATABASES;</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">+--------------------+</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">| Database |</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">+--------------------+</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">| <b>example1 </b> |</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">| information_schema |</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">| mysql |</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">| performance_schema |</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">| test |</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">+--------------------+</span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;">5 rows in set (0.01 sec)</span></div>
</div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;">If you want to setup master-master replication you have to follow steps 2 and 3, but execute the commands on the other side of the replication.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
</div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Deployment - with Juju</span></h3>
<br />
<div style="text-align: justify;">
If you haven't heart about Juju yet, now is a perfect time. Juju is an application modelling tool which works on the bases of so called charms which encapsulate the entire logic to install, configure and maintain applications. You can find more information about it at [<a href="https://jujucharms.com/">4</a>].</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Let's start with bootstrapping Juju controller on a local LXD [<a href="https://www.ubuntu.com/containers/lxd">5</a>] provider for testing purposes (if you can't install LXD on your computer, you can setup a VM with Ubuntu LTS and test from there):</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju bootstrap localhost lxd-controller</b></span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This will take a while. In meantime you can read about other providers (clouds) supported by Juju [<a href="https://docs.jujucharms.com/devel/en/clouds">6</a>]. Once the controller is bootstrapped you can start modelling. First, download the branch I created (it hasn't been merged with the upstream code yet):</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cd /tmp</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ git clone git@github.com:tytus-kurek/charm-percona-cluster.git</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span>
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ cd /tmp/charm-percona-cluster</b></span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ git checkout 1776171</b></span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Then deploy two different clusters:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju deploy --num-units 3 /tmp/charm-percona-cluster pxc1</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju deploy --num-units 3 /tmp/charm-percona-cluster pxc2</b></span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
configure them (VIPs should belong to the same subnet as lxdbr0 interface):</div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju config pxc1 root-password="changeme"</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju config pxc2 root-password="changeme"</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju config pxc1 vip="10.130.194.254"</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju config pxc1 vip="10.130.194.253"</b></span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
and deploy hacluster subordinate application for Pacemaker / Corosync management:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju deploy hacluster hacluster-pxc1</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju deploy hacluster hacluster-pxc2</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju relate pxc1 hacluster-pxc1</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju relate pxc2 hacluster-pxc2</b></span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
That's almost it! At this point you should have 2 different Percona clusters deployed. You can check the status by executing the following command:</div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju status</b></span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Assuming that all units have turned to the active status, you can enable asynchronous MySQL replication between <i>pxc1</i> and <i>pxc2</i>. First, configure clusters with mandatory settings:</div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju config pxc1 databases-to-replicate="example1,example2"</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju config pxc2 databases-to-replicate="example1,example2"</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju config pxc1 cluster-id=1</b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="font-family: "courier new" , "courier" , monospace;"><b>$ juju config pxc1 cluster-id=2</b></span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Then enable replication from <i>pxc1</i> to <i>pxc2</i> by executing the following command:</div>
<div style="text-align: justify;">
<span style="color: orange;"><br /></span></div>
<div style="text-align: justify;">
<b><span style="color: orange; font-family: "courier new" , "courier" , monospace;">$ juju relate pxc1:master pxc2:slave</span></b></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Voila! Wanna enable master-master replication? Here you go:</div>
<div style="text-align: justify;">
<span style="color: orange; font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div style="text-align: justify;">
<span style="color: orange; font-family: "courier new" , "courier" , monospace;"><b>$ juju relate pxc2:master pxc1:slave</b></span></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Don't you believe it's working? Just check it! It works like a charm. This is why I like Juju so much.<br />
<br />
P.S.: Don't hesitate to visit my next post about setting up <a href="http://tkurek.blogspot.com/2018/06/circular-asynchronous-mysql-replication.html">circular asynchronous MySQL replication between multiple geographically-distributed Percona XtraDB Clusters</a>.</div>
<h3 style="text-align: left;">
<div style="text-align: center;">
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: center;">
<span style="color: #b45f06;">References</span></div>
</h3>
<br />
[1] <a href="https://www.percona.com/resources/mysql-white-papers/high-level-multi-datacenter-mysql-high-availability">https://www.percona.com/resources/mysql-white-papers/high-level-multi-datacenter-mysql-high-availability</a><br />
<br />
[2] <a href="https://github.com/tytus-kurek/charm-percona-cluster/tree/1776171">https://github.com/tytus-kurek/charm-percona-cluster/tree/1776171</a><br />
<br />
[3] <a href="https://www.percona.com/forums/questions-discussions/percona-xtradb-cluster/9433-master-master-between-two-clusters?p=19622#post19622">https://www.percona.com/forums/questions-discussions/percona-xtradb-cluster/9433-master-master-between-two-clusters?p=19622#post19622</a><br />
<br />
[4] <a href="https://jujucharms.com/">https://jujucharms.com/</a><br />
<br />
[5] <a href="https://www.ubuntu.com/containers/lxd">https://www.ubuntu.com/containers/lxd</a><br />
<br />
[6] <a href="https://docs.jujucharms.com/devel/en/clouds">https://docs.jujucharms.com/devel/en/clouds</a></div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-73393976121922024792018-05-17T04:41:00.001-07:002018-05-17T04:47:55.629-07:00How to provide an access from an OpenStack instance to LXD container running on Neutron Gateway<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
<span style="text-align: left;">I have recently been tasked with an interesting problem. Let's assume we have an OpenStack cloud based on Open vSwitch which uses VXLAN tunneling. How would you provide an access from tenant instance to the LXD container running on Neutron Gateway node? The answer seems to be obvious - you assign floating IP to the instance and route the traffic through your networking infrastructure using OpenStack external network. But seriously? What is the point of the whole network virtualization concept then? Wouldn't it be just possible to somehow connect the tenant network with the LXD bridge on the Neutron Gateway node? The answer is "yes" and the following guide will walk you through necessary steps in order to configure it.</span><br />
<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;">Design</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<span style="font-family: inherit;">The tenant network terminates on OpenStack router which is implemented as Linux namespace on Neutron Gateway node. You can list all OpenStack routers by executing the following command:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># ip netns | grep qrouter</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">qrouter-3960f517-e2e9-4bca-900e-3681db5fe5ec</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;"><span style="font-family: inherit;">On the other side LXD containers are attached to a <b>lxdbr0</b><i> </i>bridge</span> by default. You can display it by running the following command:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># brctl show lxdbr0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">lxdbr0</span><span style="font-family: "courier new" , "courier" , monospace; white-space: pre;"> </span><span style="font-family: "courier new" , "courier" , monospace;">8000.000000000000</span><span style="font-family: "courier new" , "courier" , monospace; white-space: pre;"> </span><span style="font-family: "courier new" , "courier" , monospace;">no</span><span style="font-family: "courier new" , "courier" , monospace; white-space: pre;"> </span><span style="font-family: "courier new" , "courier" , monospace;">eth0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
Let's assume that the bridge resides on <b>10.0.0.0/24</b> subnet and the container has the IP address of <b>10.0.0.1</b>. In order to create a connection between the tenant subnet and the container we create a veth pair and assign its ends to the LXD bridge and OpenStack router.</div>
<div>
<br /></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Configuration</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
Let's start with logging to the Neutron Gateway node as the <i>root </i>user.</div>
<div>
<br /></div>
<div>
<b><span style="color: orange;">1)</span></b> We start with creation of the veth pair which will connect the LXD bridge with the OpenStack router:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># ip link add veth-qrouter type veth peer name veth-lxdbr</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">You can check whether the veth pair has actually been created by executing the following command:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># ip link | grep veth</span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">2831: <b>veth-lxdbr@veth-qrouter</b>: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">2832: <b>veth-qrouter@veth-lxdbr</b>: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000</span></div>
</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;"><b><span style="color: orange;">2) </span></b>Next we attach the <i>veth-lxdbr </i>interface to the LXD bridge and bring it up:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># brctl addif lxdbr0 veth-qrouter</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># ip link set dev veth-qrouter up</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">You can check whether it has been attached by executing the following command:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># brctl show lxdbr0</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"></span><br />
<div>
<span style="font-family: "courier new" , "courier" , monospace;">lxdbr0<span style="white-space: pre;"> </span>8000.000000000000<span style="white-space: pre;"> </span>no<span style="white-space: pre;"> </span>eth0</span></div>
<span style="font-family: "courier new" , "courier" , monospace;">
<div>
<span style="white-space: pre;"> </span><b>veth-qrouter</b><br />
<b><br /></b></div>
</span></div>
<div>
<span style="font-family: inherit;"><b><span style="color: orange;">3)</span></b> Then we attach the <i>veth-qrouter </i>interface to the OpenStack rotuer - Linux namespace:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># ip link set veth-lxdbr netns qrouter-3960f517-e2e9-4bca-900e-3681db5fe5ec</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
Let's check whether it has successfully been attached:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># ip netns exec qrouter-3960f517-e2e9-4bca-900e-3681db5fe5ec ip link </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">2: qr-c4ff6972-03@if2829: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> link/ether fa:16:3e:37:89:65 brd ff:ff:ff:ff:ff:ff</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">3: qg-55cb180e-67@if2830: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> link/ether fa:16:3e:be:7d:2d brd ff:ff:ff:ff:ff:ff</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">2831: <b>veth-lxdbr@if2832</b>: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> link/ether a2:71:8b:21:42:d8 brd ff:ff:ff:ff:ff:ff</span></div>
</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><b><br /></b></span></div>
<div>
<span style="font-family: inherit;">It is there! But wait ... it has now disappeared from the default space:</span></div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># ip link | grep veth</span></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">2832: <b>veth-qrouter@veth-lxdbr</b>: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000</span></div>
</div>
</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">Of course it did - you have just assigned it to a different space :).</span></div>
<div>
<br /></div>
<div>
<b><span style="color: orange;">4) </span></b>The interface withing the space is down. We have to bring it up and configure:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># ip netns exec qrouter-3960f517-e2e9-4bca-900e-3681db5fe5ec</span><span style="font-family: "courier new" , "courier" , monospace;"> ip addr add 10.0.0.2/24 dev veth-lxdbr</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># ip netns exec qrouter-3960f517-e2e9-4bca-900e-3681db5fe5ec ip link set dev veth-lxdbr up</span></div>
<div>
<br /></div>
<div>
Finally, we have to configure SNAT for the traffic coming from the tenant network:</div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"># ip netns exec qrouter-3960f517-e2e9-4bca-900e-3681db5fe5ec iptables -t nat -I POSTROUTING -o veth-lxdbr -j MASQUERADE </span></div>
</div>
<div>
<br /></div>
<div>
At this point tenant instances should be able to access the LXD container.</div>
</div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-72499606301326868352014-07-10T05:02:00.004-07:002019-05-10T14:17:16.299-07:00What is the cloud? - Cloud definition<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: justify;">
"What actually is the Cloud?" - I keep hearing this question everyday. The problem is that there is no single, official definition. Actually, there are as many definitions as people talking about that. The problem is that when we started using this term several years ago it had not clarified yet what does it mean and it meant a lot of different things. However, this period is over!</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
So what actually does the Cloud mean? I spent some time on clarifying it as a part of my PhD thesis and I prepared my own definition based on what I found in a number of books, articles and online resources. So basically it proceeds as follows:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="color: orange;">Cloud is an ICT (<i>Information and Communications Technology</i>) infrastructure which is characterized by the following features:</span><br />
<span style="color: orange;"><br /></span></div>
<ul>
<li style="margin-bottom: 0.1em;"><span style="color: orange;">it is completely transparent to users</span></li>
<li style="margin-bottom: 0.1em;"><span style="color: orange;">it delivers a value to users in a form of services available from the self-service portal</span></li>
<li style="margin-bottom: 0.1em;"><span style="color: orange;">its storage, network and computing resources are infinite from users' perspective apart from the configurable quotas</span></li>
<li style="margin-bottom: 0.1em;"><span style="color: orange;">it implements "pay-as-you-go" billing</span></li>
<li style="margin-bottom: 0.1em;"><span style="color: orange;">it is multi-tenant</span></li>
<li style="margin-bottom: 0.1em;"><span style="color: orange;">it is easily scalable (it scales out)</span></li>
<li style="margin-bottom: 0.1em;"><span style="color: orange;">it is highly available, often geographically-distributed</span></li>
<li style="margin-bottom: 0.1em;"><span style="color: orange;">it can run on the commodity hardware</span></li>
</ul>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Phew ;). Obviously not all clouds can be characterized by all of these features, but the most of them can. What actually is the Cloud then? I hope you will be able to answer quickly next time!</div>
<br />
<br /></div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-42273902189022416512014-01-02T04:08:00.000-08:002014-01-04T12:12:50.167-08:00MapReduce Explained<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
Have you ever thought how does Google Search engine - the core Google product that brought up the company to the position of one of the biggest, if not the biggest, leaders on ICT market - work? It all came up thanks to MapReduce data processing framework. Although nowadays Google Search leverages much more powerful engines like BigTable and Caffeine, it has it origins in MapReduce.<br />
<br />
It's everywhere around nowadays. Search engines, bank systems, colaboration platforms - they're all using MapReduce. It's <i>de facto </i>standard for processing and analysing big data sets. Any time you'll hear about Big Data, you'll also hear about MapReduce.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
But what MapReduce actually is? According to "<a href="http://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf">MapReduce: Simplied Data Processing on Large Clusters</a>" <span style="font-family: inherit;">- an official publication made by Google - a MapReduce "is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key". Although you could just read the research paper, that I do encouraged you to do anyway, I believe the best way to understand it is just to cover it based on an simple example. Lets go into the next step then where I'll show you a real, live problem where MapReduce comes with the simplest and lowest cost solution.</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;">Job</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
I live in <a href="https://www.google.pl/maps/preview#!q=krakow&data=!1m4!1m3!1d260481!2d20.0048632!3d50.0467657!4m12!2m11!1m10!1s0x471644c0354e18d1%3A0xb46bb6b576478abf!3m8!1m3!1d163975!2d20.0048633!3d50.0467657!3m2!1i1024!2i768!4f13.1">Krakow</a>, Poland that is a mid-sized EU city. Lets consider the following problem:</div>
<div>
<br /></div>
<div>
"<b><i>I need a list of all streets in Krakow, both with the highest house number on particular street. The above needs to be completed in 2 days</i></b>".</div>
<div>
<br /></div>
<div>
Sounds unworkable. I would need to visit each street, search for a house with the highest number, note it down and so on until I'll have a full list. As the overall length of streets in Krakow is around 1200 km, assuming that I would walk 15 km per day (I am not type of a sportsman and need to spend some time on looking for a house with the highest number too) the above will take me 80 days. Is there any way to accomplish the job then? The answer is: Yes, there's a solution for that and it's called MapReduce! Lets start with the Map step then.</div>
<div>
<br /></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Map</span></h3>
</div>
<div>
<br /></div>
<div>
First of all you've forgotten that I have a friends that will help me with that! I have 39 friends, so there are 40 of us. If we split the job that each of us walks 30 km per day, we will all walk 1200 km per day. That means we all could walk the whole Krakow in 1 day, so we would complete the job even before the deadline!</div>
<div>
<br /></div>
<div>
OK. But I said that I could walk 15 km per day only and lets assume that it's indeed an average of all of us. I also mentioned that this comes from both our bodies limitations - performance - and due to the fact that we need to spend some time on looking for a house with the highest number - processing time. I doubt we would be able to bypass the performance, but could we speed up processing time a bit?</div>
<div>
<br /></div>
<div>
Of course we could. Instead of looking for a house with the highest number on particular streets, we could just pass the street quickly and note down its name - key - and each house number - value. As the house number are placed in the best visible places, we would even not need to stop. Lets assume that thanks to the above we save so much time that would enable us to double the distance that we could walk, up to 30 km.</div>
<div>
<br /></div>
<div>
We all would then walk the whole Krakow in 1 day. But will the job be completed after that? Not yet as we would not have a list of streets with the highest house number, but a list of streets with all the house numbers on them instead. This is where the Reduce step comes.</div>
<div>
<br /></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Reduce</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
So far we've all spent 1 day on completing the job and we're almost done. The only task that is left is to sort the house numbers on particular streets and choose the one with the highest number. Lets assume that it's done just be me and it takes me 1 day. The job will be completed on time then!</div>
<div>
<br /></div>
<div>
But why did we hurry so? Couldn't we just walk those 15 km per day and have the job completed in 2 days anyway, without performing the Reduce step? In each case I'll need to spend 2 days on the job.</div>
<div>
<br /></div>
<div>
That's right, but you might have forgotten about my friends. They won't be involved in the Reduce step, so they'll have a day off and can help someone else, e.g in Warsaw (the capital of Poland). I'm not going to hold them anymore. Each of them did a really great task for me and we all did a really great job. Obviously once I'll sit and manually sort the house numbers that actually sounds like a nightmare ;).</div>
<div>
<br /></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Summary and Explanation</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
MapReduce is a framework for processing and generating large data sets in a quick and effective manner thanks to distributed computing paradigm. The above example shows that concept in a direct and understandable way. There's a job that gets split into 40 Map tasks and 1 Reduce task. Each task is executed either by me or my friend - tasktrackers - while the whole jobs is executed and coordinated by me only - jobtracker. After each task a list of key - value pairs gets compiled that finally leads to an ultimate one. Simple Map functions enables to speed up processing time. Assigning only one tasktracker to the Map function allows to free up the resources of the remaining tasktrackers.</div>
<div>
<br /></div>
<div>
In a real MapReduce framework jobtrackers and tasktrackers are the computer instances that cooperate together in a distributed computing paradigm. Map and Reduce are the functions written in Java, Python or actually any other language enable for data processing. The whole engine gets coordinated by a dedicated software like Apache Hadoop or proprietary software like in case of Google.</div>
<div>
<br /></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-6468327190218275532014-01-01T23:42:00.000-08:002014-01-04T06:37:05.220-08:00Starting Hadoop datanode: Error: JAVA_HOME is not set and could not be found.- Issues with CDH<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
CDH (<i>Cloudera's Distribution including apache Hadoop</i>) is the most popular and the best documented distribution of Apache Hadoop. I have recently found out some deficiencies in its <a href="http://www.cloudera.com/content/support/en/documentation/cdh4-documentation/cdh4-documentation-v4-latest.html">documentation</a> when following the <a href="http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Quick-Start/CDH4-Quick-Start.html">CDH4 Quick Start Guide</a> instructions. I installed the Oracle Java Development Kit and set up <b>JAVA_HOME </b>environmental variable according to the instructions, but when attempting to start HDFS nodes I was receiving an error message stating that <b>JAVA_HOME is not set and could not be found</b>. After a quick research I have finally found out that a solution for that is just to export <b>JAVA_HOME </b>inside <b>hadoop-env.sh </b>configuration file in addition to <b>.bash_profile </b>file. The above solution comes very quickly for an experienced Hadoop administrator, but can be tricky for a beginner, so should be well documented by Cloudera in my opinion. The following covers detailed troubleshooting steps both with a solution.<br />
<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;">Symptoms</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
1) You have the Oracle Java Development Kit installed and <b>JAVA_HOME </b>environmental variable exported according to the following <a href="http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Quick-Start/cdh4qs_topic_2_1.html">HowTo</a>:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@hadoop-standalone-mr1 ~]# env | grep JAVA_HOME</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">JAVA_HOME=/opt/jdk1.6.0_45</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
2) When attempting to start HDFS nodes you are receiving the following error messages:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">[root@hadoop-standalone-mr1 ~]# for x in `cd /etc/init.d ; ls hadoop-hdfs-*` ; do service $x start ; done</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Starting Hadoop datanode: [ OK ]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Error: JAVA_HOME is not set and could not be found.</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Starting Hadoop namenode: [ OK ]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Error: JAVA_HOME is not set and could not be found.</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Starting Hadoop secondarynamenode: [ OK ]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Error: JAVA_HOME is not set and could not be found.</span></div>
</div>
<div>
<br /></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">How to fix the issue</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
1) Export <b>JAVA_HOME</b> environmental variable in <b>hadoop-env.sh</b> configuration file:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">echo export `env | grep ^JAVA_HOME` >> /etc/alternatives/hadoop-conf/hadoop-env.sh</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
2) You should be fine. All HDFS nodes start up properly now:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-size: x-small;"><span style="font-family: Courier New, Courier, monospace;">[root@hadoop-standalone-mr1 ~]# </span><span style="font-family: 'Courier New', Courier, monospace;">for x in `cd /etc/init.d ; ls hadoop-hdfs-*` ; do service $x start ; done</span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Starting Hadoop datanode: [ OK ]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-hadoop-standalone-mr1.out</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Starting Hadoop namenode: [ OK ]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-hadoop-standalone-mr1.out</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">Starting Hadoop secondarynamenode: [ OK ]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">starting secondarynamenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-secondarynamenode-hadoop-standalone-mr1.out</span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Disclaimer</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<ul>
<li>The above has been tested on <b>CDH4 </b>package, on <b>CentOS 6.4 x86_64 </b>system, in <b>Google Compute Engine </b>environment<b>.</b></li>
<li>The above solution works both for MRv1 and YARN.</li>
</ul>
<div>
<br /></div>
</div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com2tag:blogger.com,1999:blog-4818704452869317626.post-86471711834322121462014-01-01T22:59:00.000-08:002014-01-01T22:59:10.885-08:00How to connect Thunderbird to Exchange - DavMail Server<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
Mozilla Thunderbird, one of the most popular email clients, still suffers from one serious disease: it does provide neither built-in mechanisms nor third-party plugins for RPC MAPI connection to MS Exchange Server. As an email services administrator I always used to support my customers by enabling direct SMTP / IMAP connection then. However as my company has recently changed its security policy and decided to block raw SMTP / IMAP access to our MS Exchange infrastructure I was forced to find out an alternative solution for my Thunderbird users. After hours spent on digging for a best possible solution I have finally found <a href="http://davmail.sourceforge.net/">DavMail</a>. It occurred that it is some kind of proxy, written in Java, that runs SMTP / IMAP servers locally and connects to MS Exchange via OWA. I managed to run DavMail in a server mode on a standalone VM. After that I provided my Thunderbird users with the VM details and now I have all of them connected to company MS Exchange infrastructure. The following HowTo presents details steps describing how did I achieve that.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Installation</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
To set up the DavMail in a server mode on a VM follow the instructions below:</div>
<div>
<br /></div>
<div>
1) Set up a VM with Ubuntu Server 12.04 64-bit with X Server (Unity preferably).</div>
<div>
<br /></div>
<div>
2) Install OpenJDK and SWT by running the following command:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># apt-get install openjdk-7-jre libswt-gtk-3-java</span></div>
<div>
<br /></div>
<div>
3) Download the newest available version of <a href="http://sourceforge.net/projects/davmail/files/davmail/">DavMail</a> and install it both with required dependencies by issuing the following command:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># dpkg -i davmail*.deb</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># apt-get -f install</span></div>
<div>
<br /></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Configuration</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
1) To run DavMail in a server mode create <b>/etc/davmail </b>directory and put <b><a href="http://davmail.sourceforge.net/serversetup.html">davmail.properties</a> </b>file there. Adjust the following settings to fit into your organization requirements:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">davmail.url=https://mail.mycompany.com/owa/</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">davmail.caldavPort=443</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">davmail.imapPort=993</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">davmail.ldapPort=636</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">#davmail.popPort=995</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">davmail.smtpPort=465</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">davmail.allowRemote=true</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">The most important ones are <b>davmail.url </b>that indicates your <b>OWA URL</b> and <b>davmail.allowRemote </b>that you need to turn to <b>true </b>to support server mode. Moreover in my case I also disabled POP server and changed <b>davmail.caldavPort</b>, <b>davmail.imapPort</b>, <b>davmail.ldapPort </b>and <b>davmail.smtpPort </b>values into regular port numbers of <b>HTTPS</b>, <b>IMAPS</b>, <b>LDAPS</b>, and <b>SMTPS </b>services respectively.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">2) As all of the <b>HTTPS</b>, <b>IMAPS</b>, <b>LDAPS</b> and <b>SMTPS </b>services run over <b>TLS </b>you will need a certificate in <b>PKCS12 </b>format attached. To generate it, assuming that you have the following in a <b>PEM</b> format: <b>CA.pem</b>, <b>server.pem</b> and <b>server.key</b> run the following command:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">openssl pkcs12 -export -in server.pem -inkey server.key -certfile CA.pem -out server.p12</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">Alternatively you can create a <a href="http://support.nordicedge.com/nsd1309-creating-self-signed-certificates-using-openssl/">self-signed certificate</a> or not attach it at all. Your setup will not be secure then, so it is highly recommended to use <b>TLS </b>anyway.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">3) Adjust the following settings of <b>davmail.properties </b>configuration file:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">davmail.ssl.keystoreType=PKCS12</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">davmail.ssl.keystoreFile=/etc/ssl/certs/server.p12</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">davmail.ssl.keystorePass=keystorePasswordFromPreviousSection</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">davmail.ssl.keyPass=keyPasswordProvidedWhenCreatingTheKey</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Running DavMail Server</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
To run DavMail add the following line into the <b>/etc/rc.local </b>script before the <b>exit 0 </b>line:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">nohup /usr/bin/davmail /etc/davmail/davmail.properties &</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">After that you will notice that you VM starts listening on TCP ports <b>443</b>, <b>465</b>, <b>636</b> and <b>993</b>. Follow the instructions on official DavMail <a href="http://davmail.sourceforge.net/thunderbirdimapmailsetup.html">website</a> to configure your Thunderbird client.</span></div>
<div>
<br /></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-5854405027258260882013-10-16T02:52:00.002-07:002013-10-16T02:52:40.901-07:00Type: VPN Subtype: encrypt Result: DROP - asymmetric ACLs on Cisco IPsec VPN ASA edges<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
I have recently encountered some strange issue as for the Cisco IPsec VPN between two sites of my organization. I had the VPN SAs established and proper ACLs permitting a desired subset of a traffic attached to the crypto map, but the traffic was not able to pass anyway. Finally, it has been revealed that this was happening because of asymmetric ACLs on the neighboring Cisco ASAs.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Lets now have a look on a simplified configuration I had. On the site A I had subnets from 10.0.0.0/8 network. On the site B I had subnets from 172.16.0.0/12 network. Cisco ASA on the site A was running Cisco Adaptive Security Appliance Software Version 8.2(5) while Cisco ASA on the site B was running version 7.2.(4) (I suppose that the issue might be related to software versions incompatibility, a bug in a certain software version, etc.). Related part of configuration on each of the Cisco ASA instances was as follows:</div>
<div style="text-align: justify;">
<ul>
<li><b>site A:</b></li>
</ul>
<div>
<span style="font-family: Courier New, Courier, monospace;">...</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">object-group network A-site-subnets</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> network-object 10.0.0.0 255.255.0.0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> network-object 10.1.0.0 255.255.0.0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> ...</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> network-object 10.255.0.0 255.255.0.0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">object-group network B-site-subnets</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> network-object 172.16.0.0 255.240.0.0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">...</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">access-list A2B extended permit ip object-group A-site-subnets object-group B-site-subnets</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">...</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">crypto map A-map 1 match address A2B</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">...</span></div>
<ul>
<li><b>site B:</b></li>
</ul>
<div>
<span style="font-family: Courier New, Courier, monospace;">...</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">object-group network B-site-subnets</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> network-object 172.16.0.0 255.255.0.0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> network-object 172.17.0.0 255.255.0.0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> ...</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> network-object 172.31.0.0 255.255.0.0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">object-group network A-site-subnets</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> network-object 10.0.0.0 255.0.0.0</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">...</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">access-list B2A extended permit ip object-group B-site-subnets object-group A-site-subnets</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">...</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">crypto map B-map 1 match address B2A</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">...</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Troubleshooting</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
To quickly test whether the communication issues are caused by asymmetric ACLs on the neighboring Cisco ASA edges, run the <b>packet-tracer</b> command on any instance specifying the parameters that should result in an <b>ALLOW </b>decision:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">A-site-ASA# packet-tracer input inside icmp 10.0.0.1 0 0 172.16.0.1</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">...</span></div>
<div>
<div style="font-family: 'Courier New', Courier, monospace;">
Phase: 11 </div>
<div style="font-family: 'Courier New', Courier, monospace;">
Type: VPN</div>
<div style="font-family: 'Courier New', Courier, monospace;">
Subtype: encrypt</div>
<div style="font-family: 'Courier New', Courier, monospace;">
Result: DROP</div>
<div style="font-family: 'Courier New', Courier, monospace;">
Config:</div>
<div style="font-family: 'Courier New', Courier, monospace;">
Additional Information:</div>
<div style="font-family: 'Courier New', Courier, monospace;">
<br /></div>
<div style="font-family: 'Courier New', Courier, monospace;">
Result:</div>
<div style="font-family: 'Courier New', Courier, monospace;">
input-interface: inside</div>
<div style="font-family: 'Courier New', Courier, monospace;">
input-status: up</div>
<div style="font-family: 'Courier New', Courier, monospace;">
input-line-status: up</div>
<div style="font-family: 'Courier New', Courier, monospace;">
output-interface: outside</div>
<div style="font-family: 'Courier New', Courier, monospace;">
output-status: up</div>
<div style="font-family: 'Courier New', Courier, monospace;">
output-line-status: up</div>
<div style="font-family: 'Courier New', Courier, monospace;">
Action: drop</div>
<div style="font-family: 'Courier New', Courier, monospace;">
Drop-reason: (acl-drop) Flow is denied by configured rule</div>
<div style="font-family: 'Courier New', Courier, monospace;">
<br /></div>
<div>
<span style="font-family: inherit;">If the command results in an output like the above, you can safely move to the following section.</span></div>
</div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">How to fix the issue</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
To get rid of the above errors redesign A-site-subnets and B-site-subnets object groups, and as a result the A2B and B2A ACLs, that they either include the particular subnets or the whole network summary. To save my time I have chosen the second approach:</div>
<div>
<ul>
<li><b>site A:</b></li>
</ul>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">object-group network A-site-subnets</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> network-object 10.0.0.0 255.0.0.0</span></div>
</div>
<ul>
<li><b>site B:</b></li>
</ul>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">object-group network B-site-subnets</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> network-object 172.16.0.0 255.248.0.0</span></div>
</div>
</div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">A this point the traffic should be able to pass between Cisco ASA instances.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com6tag:blogger.com,1999:blog-4818704452869317626.post-58598975565538921982013-10-16T01:17:00.003-07:002013-10-16T01:35:39.493-07:00The DFS replication service stopped replication on the replicated folder at local path ... - complex DFS issues<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
I have recently performed an upgrade of the DFS infrastructure at my company that consists of 2 servers from which one of them is a master and the other is a slave for DFS Replication service. As I needed to replace the disks on the slave node and had lost the replicated data permanently as a result, I configured the DFS services from scratch then and started the replication over again. Unfortunately, after a long period of time spent on wondering whether the data are being replicated or not, I have finally found the following message marked as Warning in Event Viewer on the master node:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
"<i><b>The DFS replication service stopped replication on the replicated folder at local path ... </b></i>"</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
The following article presents how have I bypassed the above issues thanks to the articles on <a href="http://social.technet.microsoft.com/Forums/windowsserver/en-US/a16f5f7a-e748-4da0-aee4-159c6220af9a/windows-2008-dfs-initial-replication-not-completing-this-member-is-waiting-for-initial-replication?forum=winserverfiles">Technet</a> and one of the Internet <a href="http://icantbelieveidothiswithmyfreetime.wordpress.com/2012/02/13/the-source-file-names-are-larger-than-supported-by-the-file-system/">blogs</a>. I have also included some of my own augmentations into the provided solutions. Hope you'll find those information useful and consolidated. </div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">How to fix the issue</span></h3>
</div>
<div style="text-align: justify;">
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
Following the instructions on <a href="http://social.technet.microsoft.com/Forums/windowsserver/en-US/a16f5f7a-e748-4da0-aee4-159c6220af9a/windows-2008-dfs-initial-replication-not-completing-this-member-is-waiting-for-initial-replication?forum=winserverfiles">Technet</a>:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
1) Stop and disable <b>DFS Replication</b> service.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
2) Go into the drive containing replicated folder and make sure that the following folder options are set as follows:</div>
<div>
<ul>
<li style="text-align: justify;"><b>Show hidden files, folders, and drives - ENABLED</b></li>
<li style="text-align: justify;"><b>Hide protected operating system files (Recommended) - DISABLED</b></li>
</ul>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
3) On the <b>Security</b> tab of <b>System Volume Information</b> folder <b>Properties</b> add the user that you're currently logged in with <b>Full Control</b> permissions and the scope of <b>This folder only</b>.</div>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
4) Go into <b>System Volume Information</b> folder and on the <b>Security</b> tab of <b>DFSR</b> folder <b>Properties </b>add the user that you're currently logged in with <b>Full Control</b> permission and the scope of T<b>his folder, subfolders and files.</b> Make sure that permissions get propated to all child items of the folder.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
5) Remove the <b>DFSR</b> folder. If the above results in the <b>Source Path Too Long</b> error like it's shown on the following screenshot:</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-OvRYNmvCB7U/Ul5FQArfo-I/AAAAAAAAubg/v_hiLZ3NAXo/s1600/Spirce+Path+Too+Long.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="The source file name(s) are larger than is supported by the file system. Try moving to a location which has a shorter path name, or try renaming to shorter name(s) before attempting this operation." border="0" src="http://4.bp.blogspot.com/-OvRYNmvCB7U/Ul5FQArfo-I/AAAAAAAAubg/v_hiLZ3NAXo/s1600/Spirce+Path+Too+Long.png" title="Source Path Too Long" /></a></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
perform the following steps (thanks to this <a href="http://icantbelieveidothiswithmyfreetime.wordpress.com/2012/02/13/the-source-file-names-are-larger-than-supported-by-the-file-system/">blog</a>):</div>
<div style="text-align: justify;">
<ul>
<li>create <b>TEMP</b> folder in the <b>C</b> drive root</li>
<li>run the following command</li>
</ul>
<div>
<span style="font-family: Courier New, Courier, monospace;"> robocopy C:\TEMP [DFSR folder path] /MIR</span></div>
<div>
<ul>
<li>remove both <b>TEMP</b> and <b>DFSR</b> folders, this time fortunately without the above error.</li>
</ul>
<div>
<br /></div>
<div>
6) Remove the user that you're currently logged in from <b>Security</b> tab of <b>System Volume Information</b> folder <b>Properties</b>.</div>
</div>
<div>
<br /></div>
<div>
7) Enable the <b>DFS Replication</b> service and start it back. You're done. The DFS replication starts over again!</div>
<div>
<br /></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com1tag:blogger.com,1999:blog-4818704452869317626.post-28930510079261123352013-08-15T23:03:00.001-07:002013-08-15T23:03:26.218-07:00DFS folder deeply hidden, invisible<h3 style="text-align: center;">
<span style="color: #b45f06;">Case</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
It's another time when I'm working on Windows Server 2012 and discover something really odd and poorly documented. This time it was about DFS replication. I set it up between two servers in two remote AD sites. All according to regular manuals. It was working fine: the folder was claimed to be replicated, there were no issues reported by the Event Viewer and I was able to mount the share associated with the replicated folder on the remote server So what was wrong? <b>The folder was invisible in the file system on the remote server</b>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I was able to access it in the Windows Explorer only when specifying a full path into it. The content was just displayed fine then. The same as for the command line, even when running it as the Administrator user. After disabling the DFS replication the folder was still invisible. So what was wrong? After deep troubleshooting and looking for a solution in the Internet I finally found a clue that the <b>DFS replicated folders have hidden and system attributes set up by default</b>. That's why the don't appear in the filesystem! That's what's called deeply hidden folders.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Solution</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
In order to make the DFS replicated folders visible in the filesystem back, type the following command from the Windows CLI:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">attrib -r -h -s [path to the folder]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">Well done! The folder is visible in the filesystem back. Just not sure why isn't it clearly stated and documented? I hope that the above quick CaseStudy will help some abashed sysadmins like me to save a lot of time spent on troubleshooting strange behavior of Microsoft products.</span></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com1tag:blogger.com,1999:blog-4818704452869317626.post-1094991120421783122013-08-07T12:47:00.000-07:002013-08-07T12:47:51.359-07:00Sventon looses configuration after Tomcat restart on Ubuntu<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
<a href="http://www.sventon.org/">Sventon</a> is a web GUI for SVN repositories browsing. I have recently set it up for the Engineering team in my company. I configured it according to official <a href="https://code.google.com/p/sventon/wiki/Start">instructions</a> and everything was working just fine. But then, <b>after restarting the underlying Tomcat servlet container my configuration was lost</b>! How come?</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I spent a lot of time troubleshooting that and it has finally occured that the <b>Sventon by default stores its configuration in Tomcat servlet container temp directory</b>. What an absurdity? It's obvious that the configuration will be lost each time you restart the Tomcat servlet container now.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I've then found couple of topics on the Sventon support <a href="https://groups.google.com/forum/#!forum/sventon-support">forum</a> regarding the above issue, but it looks like its developers doesn't consider it as a bug claiming that different servlet container may behave diversely. The only think that I can do then is to share the exact instruction of how to bypass the above issue hoping that it will save hours of work of some who read it.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Configuration</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
Once you've installed and configured Sventon according to the following <a href="https://code.google.com/p/sventon/wiki/Start">instructions</a>, please use the below steps to save its configuration in other location than Tomcat servlet container temp directory:</div>
<div>
<br /></div>
<div>
1) Edit the <b>applicationContext.xml </b>file inside the Tomcat servlet container <b>webapps/svn/WEB-INF </b>(<b>/var/lib/tomcat7/webapps/svn/WEB-INF/applicationContext.xml </b>in my case)<b> </b>directory and change the following line:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><bean id="sventonTempRootDir" class="java.io.File"></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">to:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"><bean id="sventonTempRootDir" class="/etc/"></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">2) Create <b>/etc/sventon_config</b> directory and assign RW right for the user that the Tomcat servlet container runs in the system.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">3) Copy the content of Tomcat servlet container temp directory (<b>/tmp/tomcat7-tomcat7-tmp/sventon_config/ </b>in my case) into the newly created directory. Make sure that you have a proper permissions set up.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">You can safely restart the Tomcat servlet container now.</span></div>
</div>
<div style="text-align: justify;">
<br /></div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-55273414498996779582013-07-19T11:42:00.001-07:002013-07-19T11:46:06.361-07:00Tomcat 7 HTTP to HTTPS redirect<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<h3 style="text-align: center;">
<div>
<br /></div>
<div style="text-align: justify;">
<div>
<span style="font-weight: normal;"><span style="font-family: inherit; font-size: small;">The following article shows how to easily redirect HTTP to HTTP in Tomcat 7 servlet container that it always requires secure connection. It was assumed that the following TCP ports are used for that purpose:</span></span></div>
<div>
<ul>
<li><span style="font-family: inherit; font-size: small;">8080</span><span style="font-family: inherit; font-size: small; font-weight: normal;">: for HTTP</span></li>
<li><span style="font-family: inherit; font-size: small;">8443</span><span style="font-family: inherit; font-size: small; font-weight: normal;">: for HTTPS </span></li>
</ul>
</div>
<div>
<span style="font-weight: normal;"><span style="font-family: inherit; font-size: small;">Please, follow the exact steps as described below to get it done.</span></span></div>
<div>
<span style="font-weight: normal;"><span style="font-family: inherit; font-size: small;"><br /></span></span></div>
</div>
</h3>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Configuration</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
1) <b style="text-align: justify;">Update server.xml configuration file </b><span style="text-align: justify;">in Tomcat home directory</span><span style="text-align: justify;"> and change the following part of its configuration:</span></div>
<div style="text-align: justify;">
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: 'Courier New', Courier, monospace;"><Connector port="8080" protocol="HTTP/1.1"</span></div>
<div style="text-align: justify;">
<span style="text-align: justify;"></span><br />
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="text-align: justify;"> connectionTimeout="20000"</span></div>
<span style="text-align: justify;">
</span>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="text-align: justify;"> URIEncoding="UTF-8"</span></div>
<span style="text-align: justify;">
<div style="font-family: 'Courier New', Courier, monospace;">
redirectPort="8443" /></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<br /></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-family: 'Times New Roman';">to what's shown below:</span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<span style="font-family: 'Times New Roman';"><br /></span></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<Connector port="8080" enableLookups="false"</div>
<div style="font-family: 'Courier New', Courier, monospace;">
redirectPort="8443" /></div>
<div style="font-family: 'Courier New', Courier, monospace;">
<br /></div>
<div>
<span style="font-family: inherit;">2) <b>Update web.xml configuration file</b> in Tomcat home directory and add the following content into the end <b>before the closing </web-app> markup</b>:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><security-constraint></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><web-resource-collection></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><web-resource-name>Protected Context</web-resource-name></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><url-pattern>/*</url-pattern></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"></web-resource-collection></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><!-- auth-constraint goes here if you requre authentication --></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><user-data-constraint></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><transport-guarantee>CONFIDENTIAL</transport-guarantee></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"></user-data-constraint></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"></security-constraint></span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">3) <b>R</b></span><b style="text-align: left;">estart Tomcat </b><span style="text-align: left;">servlet container.</span></div>
<div>
<span style="text-align: left;"><br /></span></div>
<div>
<div style="text-align: left;">
<span style="font-family: inherit;">You're done! The Tomcat always requires secure connection now.</span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: left;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Related topics:</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<span style="color: #b45f06;">http://tkurek.blogspot.com/2013/07/how-to-secure-tomcat-7-with-ssl-tls.html</span></div>
</div>
</div>
</span></div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com28tag:blogger.com,1999:blog-4818704452869317626.post-52225193455409589352013-07-19T11:42:00.000-07:002013-10-06T22:01:07.032-07:00How to secure Tomcat 7 with SSL / TLS<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
The following article shows how to secure Tomcat 7 servlet container with SSL / TLS. Although there might me numerous different solutions (e.g. proxying from Apache server) the one that I present bases on Tomcat only and utilizes its default configuration files. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The following assumptions have been made for the rest of the article:</div>
<div style="text-align: justify;">
<ul>
<li><b>OS</b>: Ubuntu 12.04 Server x64</li>
<li><b>Tomcat</b>: tomcat7 installed from official Ubuntu repositories (apt-get install tomcat7)</li>
<li><b>Tomcat user: </b>tomcat7</li>
<li><b>Tomcat home directory</b>: /etc/tomcat7</li>
<li><b>SSL / TLS port</b>: TCP port 8443</li>
<li><b>Keystore location</b>: /etc/tomcat7/keystore.jks</li>
<li><b>Keystore password</b>: keystore</li>
</ul>
<div>
The following section describes a detailed steps required for securing Tomcat 7 with SSL / TLS. All the commands have been run as a tomcat7 user.</div>
</div>
<div style="text-align: justify;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Configuration</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
1) <b>Generate the keystore file</b> that will store the certificates trusted by the Tomcat server. Depending on your needs this step may require invoking different commands. A general HowTo regarding the keytool tool usage can be found <a href="https://www.sslshopper.com/article-most-common-java-keytool-keystore-commands.html">here</a>.</div>
<div>
<br /></div>
<div>
In my case the keystore file was already delivered to me by my CA when requesting the certificate. However having both the CA and the CA-signed certificate you can easily create the keystore file by running the following commands:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span style="text-align: left;">keytool -import -trustcacerts -alias root</span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span style="text-align: left;">-file [CA cert path]</span><span style="text-align: left;"> </span></span><span style="font-family: 'Courier New', Courier, monospace; text-align: left;">-</span><span style="font-family: 'Courier New', Courier, monospace; text-align: left;">keystore /etc/tomcat7/keystore.jks</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span style="text-align: left;"><br /></span></span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">keytool -import -trustcacerts -alias tomcat</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">-file [CA-signed cert path] -keystore /etc/tomcat7/keystore.jks</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: inherit;">Or to generate the self-signed certificate:</span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">keytool -genkey -keyalg RSA -alias tomcat</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">-keystore /etc/tomcat7/keystore.jks -storepass keystore</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">-validity 360 </span><span style="font-family: 'Courier New', Courier, monospace;">-keysize 2048</span></div>
<div>
<br /></div>
<div>
2) <b>Update /etc/tomcat7/server.xml configuration file</b> and change the following part of its configuration:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><!--</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> maxThreads="150" scheme="https" secure="true"</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> clientAuth="false" sslProtocol="TLS" /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"> --></span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">to what's shown below:</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; text-align: left;"><Connector protocol="org.apache.coyote.http11.Http11NioProtocol"</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; text-align: left;"> port="8443" SSLEnabled="true" maxThreads="200"</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; text-align: left;"> scheme="https" secure="true"</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; text-align: left;"> keystoreFile="/etc/tomcat7/keystore.jks"</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; text-align: left;"> keystorePass="keystore" clientAuth="false"</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace; text-align: left;"> sslProtocol="TLS" /></span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: inherit;">3) <b>As a root user restart Tomcat</b> by running the following command:</span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">/etc/init.d/tomcat7 restart</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: inherit;">You're done! The Tomcat is now secured with SSL / TLS on port 8443.</span></div>
<div style="text-align: left;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: left;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Related topics:</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<span style="color: #b45f06;">http://tkurek.blogspot.com/2013/07/tomcat-7-http-to-https-redirect.html</span></div>
</div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com14tag:blogger.com,1999:blog-4818704452869317626.post-32379268151377179012013-07-12T21:10:00.000-07:002013-07-12T21:10:17.014-07:00How to view JSESSIONID of the current session<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
JSESSIONID is a cookie created by the servlet engine after successful authentication attempt and used for session management in JSP applications for HTTP. But how to view it? After deep research I have found 3 easy ways. First two are CLI-based and are designed to the Linux systems only. Those utilize <b>curl </b>and <b>wget </b>commands. The last one is platform independent and utilizes <b>Mozilla Firefox Live HTTP headers </b>extension. The following section describes detailed instructions of how to accomplish that using each of the above solutions.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">curl</span></h3>
</div>
<div style="text-align: justify;">
With curl you only need to run a single command that will attempt an authentication against the specified URL. The corresponding JSESSIONID will be displayed as part of a result:</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;">curl -u [login]:[password] '[URL]' --head</span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"></span><br />
<div>
<span style="font-family: Courier New, Courier, monospace;">HTTP/1.1 302 Moved Temporarily</span></div>
<span style="font-family: Courier New, Courier, monospace;">
</span>
<div>
<span style="font-family: Courier New, Courier, monospace;">Date: Mon, 08 Jul 2013 05:53:18 GMT</span></div>
<span style="font-family: Courier New, Courier, monospace;">
<div>
X-AREQUESTID: 113x7408260x1</div>
<div>
Set-Cookie: JSESSIONID=<b>30C6F08DFEF05B08961C503DFCE2D88E</b>; Path=/; Secure; HttpOnly</div>
<div>
X-Seraph-LoginReason: OK</div>
<div>
X-ASESSIONID: 1j16vj3</div>
<div>
X-AUSERNAME: myuser</div>
<div>
X-Content-Type-Options: nosniff</div>
<div>
Location: http://mywebsite.com/SomeApp.jsp</div>
<div>
Content-Type: text/html;charset=UTF-8</div>
<div>
Vary: Accept-Encoding</div>
</span></div>
<div style="text-align: justify;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">wget</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
1) Gather the information required for CLI-based authentication. It means that you need to know the authentication form fields names. You may check that by viewing the web page source and analyzing the HTML code. The following example assumes that the fields are called <b>login </b>and <b>password </b>respectively.</div>
<div>
<br /></div>
<div>
2) Execute the following command that will attempt an authentication against the specified URL and create <b>cookies.txt </b>file containing the JSESSIONID:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">wget --cookies=on --keep-session-cookies --save-cookies cookies.txt --post-data 'login=[login]&password=[password]' [URL]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">3) You can find the JSESSIONID inside the <b>cookies.txt </b>file:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">mywebsite.com FALSE / TRUE 0 JSESSIONID <b>7E830440B8EB9D1A43DA1452881285AE</b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b><br /></b></span></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Mozilla Firefox Live HTTP headers extension</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
1) From the Firefox menu select the <b>Tools </b>bar and then select the <b>Add-ons.</b></div>
<div>
<br /></div>
<div>
2) Search for the <b>Live HTTP Headers </b>extension and then install it:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-mxcbmZd6EVo/Ud-kwrH4YOI/AAAAAAAAt9w/jOtyKGFKEnQ/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" height="115" src="http://2.bp.blogspot.com/-mxcbmZd6EVo/Ud-kwrH4YOI/AAAAAAAAt9w/jOtyKGFKEnQ/s640/1.png" title="How to view JSESSIONID of the current session" width="640" /></a></div>
<div>
<br /></div>
<div>
3) Restart the Firefox after it downloads the extension:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-tPLy246CUXw/Ud-k2_aqJvI/AAAAAAAAt94/KdJdW6ltAYY/s1600/2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" height="124" src="http://2.bp.blogspot.com/-tPLy246CUXw/Ud-k2_aqJvI/AAAAAAAAt94/KdJdW6ltAYY/s640/2.png" title="How to view JSESSIONID of the current session" width="640" /></a></div>
<div>
<br /></div>
<div>
4) You will find the <b>Live HTTP Headers </b>on a list of your extensions installed:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-_HqB7PnJ9Is/Ud-k7mB9I2I/AAAAAAAAt-A/paN6COcE-dg/s1600/3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" height="114" src="http://3.bp.blogspot.com/-_HqB7PnJ9Is/Ud-k7mB9I2I/AAAAAAAAt-A/paN6COcE-dg/s640/3.png" title="How to view JSESSIONID of the current session" width="640" /></a></div>
<div>
<br /></div>
<div>
5) Press the <b>Preferences </b>button. A new window appears. Make sure that the <b>Capture </b>check box is checked:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-HTdDSZFf5pg/Ud-k_QQeScI/AAAAAAAAt-I/r2Q9aY8BO_I/s1600/4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" height="449" src="http://3.bp.blogspot.com/-HTdDSZFf5pg/Ud-k_QQeScI/AAAAAAAAt-I/r2Q9aY8BO_I/s640/4.png" title="How to view JSESSIONID of the current session" width="640" /></a></div>
<div>
<br /></div>
<div>
6) Log into the website. All headers will be captured. Just search for the <b>JSESSIONID</b>:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-mDIwuzAZGkM/Ud-lEU37wxI/AAAAAAAAt-Q/qOa9yZ4tMxw/s1600/5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" height="450" src="http://2.bp.blogspot.com/-mDIwuzAZGkM/Ud-lEU37wxI/AAAAAAAAt-Q/qOa9yZ4tMxw/s640/5.png" title="How to view JSESSIONID of the current session" width="640" /></a></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-59998544099075374202013-06-23T23:10:00.000-07:002013-06-23T23:10:51.963-07:00%252F instead of %2F in URL - Apache rewrite module and NE flag<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
I have been recently migrating some of my company data from one Apache server into another with different configuration. After moving the web content I noticed that some of the links on the website are broken. After detailed troubleshooting I finally compared them with the original ones and it occurred that what's causing the issue was the <b>%252F </b>code in URL instead of <b>%2F</b>. After some hours spent on searching solution in the Internet I have finally managed to fix the above issue by setting up the <b>NE flag </b>in Apache <b>RewriteRule </b>directive. However the above is fully sufficient to make it working and you can stop here, please find the detailed description of the issue both with the solution in the following sections.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Case</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
1) You are using the <b>Apache</b> server with <b>rewrite module</b> enabled and <b>Rewrite directives</b> inside its configuration files.</div>
<div>
<br /></div>
<div>
2) Some of the URLs get broken that the <b>%252F </b>code is being displayed in a browser instead of expected <b>%2F </b>code.</div>
<div>
<br /></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Cause</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
The <b>%2F </b>is an encoded forward slash (<b>/</b>) while the <b>%252F </b>is double encoded forward slash. This happens because of using the<b> </b>rewrite module that the Apache first encodes the forward slash characters and then encodes them one more time by default. As a result the browser refers to the directory on the Apache server that does not exist as the directories paths are separated by '<b>/</b>' characters in file system instead of '<b>%2F</b>' characters, aren't they?</div>
<div>
<br /></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Solution</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
There is a quick solution for the above issues. All you need to do is to add the <a href="http://httpd.apache.org/docs/current/rewrite/flags.html#flag_ne">NE</a> flag into the Apache <b>RewriteRule </b>directive. The <b>NE </b>flag preserves special characters (i.e. '<b>/</b>' character) from being encoded resulting in '<b>/</b>' being encoded only once. The following is an example of such a directive that I used on my Apache web server:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">RewriteRule ^/$ https://www.domain.com/files/ [<b>NE</b>,R,L]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">The above redirects browsers connecting into the www.domain.com website into the 'files' directory.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Summary</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
The above issues is not a bug, but it is rather a bad design. You need to change the default behavior of the rewrite engine to make it working. I hope that you will find this article useful.</div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com10tag:blogger.com,1999:blog-4818704452869317626.post-58955362108132030832013-06-21T11:33:00.000-07:002013-06-21T11:33:54.131-07:00How to upgrade Dell BIOS on Ubuntu<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
According to <a href="https://wiki.ubuntu.com/DellBIOS">Ubuntu Wiki</a> Dell has discontinued support for Linux. Just such a "nice" surprise. How to upgrade the BIOS then when all of the files are now distributed as a Windows executables? Fortunately, before trying any of not so easily looking solutions on the <a href="https://wiki.ubuntu.com/DellBIOS">Ubuntu Wiki</a> I have found out some other way to do it. The following section describes the detailed steps required to upgrade the BIOS on Ubuntu.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Upgrade</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
1) First of all, you need to install the required package:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ sudo apt-get install smbios-utils</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">2) The above package package provides the <b>getSystemId </b>command that we can use to verify the System ID of the machine that is a hex value that is unique for any of the Dell products or products family. I.e. when running the command on my <b>PowerEdge R300 </b>the following output was returned:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ sudo getSystemId </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Libsmbios version: 2.2.28</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Product Name: PowerEdge R300</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Vendor: Dell Inc.</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>BIOS Version: 1.5.1</b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>System ID: 0x020F</b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Service Tag: ABCDEFG</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Express Service Code: 13772621908</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Asset Tag: </span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Property Ownership Tag:</span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">3) Now you need to go into the following <a href="http://linux.dell.com/repo/firmware/bios-hdrs/">website</a>, find out the directory containing the System ID that you've just found in its name and use the one that has the latest BIOS version in its name too. I.e. for the above machine this was the following one:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">system_bios_ven_0x1028_dev_<b>0x020f</b>_version_<b>1.5.2</b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b><br /></b></span></div>
<div>
<span style="font-family: inherit;">5) Go into the directory and download the <b>bios.hdr </b>file. Put it somewhere on your machine and launch the following command to start the BIOS upgrade process:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ sudo modprobe dell_rbu</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ sudo dellBiosUpdate -u -f bios.hdr</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">6) Reboot your machine. It should come up with the BIOS version that you've already downloaded and installed. You may check that by running the following command after reboot:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">$ sudo dmidecode -s bios-version</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>1.5.2</b></span></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com12tag:blogger.com,1999:blog-4818704452869317626.post-82861856104796780712013-05-27T11:10:00.000-07:002013-05-27T11:10:19.543-07:00IIS AppPool Identities - IIS 8 permissions issues<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
Recently I have experienced some strange permission issues when setting up <b>IIS 8</b> on <b>Windows Server 2012</b>. It occurred that on the default setting all AD users can log in into the hosted website even if they are not explicitly listed in the file system permissions. After long troubleshooting process I have finally managed to fix the above issue anyway using the new <b>IIS </b>feature - <b>AppPool Identities</b>.<br />
<br />
The server used in the following scenario was called <b>BLINFS01 </b>and my account, that appears in the pictures, was called <b>Tytus Kurek</b>. I am not sure whether the described issue is a bug or not, but it is definitely something odd and I could not find any explanation when looking for a solution on the Microsoft sites and forums.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Initial Setup</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
After installing <b>IIS 8 </b>from <b>Server Manager</b> I have performed the following steps:</div>
<div>
<br /></div>
<div>
1) I created a regular directory on a separate drive that was going to be hosted by <b>IIS</b>. The directory automatically inherited permissions as shown on the attached picture:<br />
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-YFr39Va3btY/UaMHRJUCuNI/AAAAAAAAtpQ/eXJDsB5_Eug/s1600/blog1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" height="492" src="http://4.bp.blogspot.com/-YFr39Va3btY/UaMHRJUCuNI/AAAAAAAAtpQ/eXJDsB5_Eug/s640/blog1.png" title="IIS 8 permissions issues" width="640" /></a></div>
<br /></div>
<div>
where <b>Special </b>permissions for <b>BLINFS01\Users </b>group were:</div>
<div>
<ul>
<li><b>Create files / write data</b></li>
<li><b>Create folders / append data</b></li>
</ul>
<div>
2) I created a regular site in <b>IIS Manager </b>hosting the directory. Then in site configuration window I enabled <b>Windows Authentication </b>and disabled <b>Anonymous Authentication </b>that had been enabled by default as shown on the attached picture:<br />
<br /></div>
</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-Iyn2AGIDgrc/UaMHgBkTOrI/AAAAAAAAtpY/OU-_eFbHkl8/s1600/blog2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" height="498" src="http://4.bp.blogspot.com/-Iyn2AGIDgrc/UaMHgBkTOrI/AAAAAAAAtpY/OU-_eFbHkl8/s640/blog2.png" title="IIS 8 permissions issues" width="640" /></a></div>
<br /></div>
<div>
After completing the above steps I restarted <b>IIS</b> service and I was able to successfully login into my website using my regular AD credentials.</div>
<div>
<br /></div>
<div>
That was great, but shortly it was revealed that anyone in the domain can log in into the website too. If you carefully had a look on the picture provided in point 1 then you might probably notice that there's not way for that. The only domain account that's listed in the permissions lap is my account.</div>
<div>
<br /></div>
<div>
I found <b>Effective Access </b>lap very useful in the troubleshooting process. It allows you to test permissions for any user and shows the results. Soon I found that after removing permissions for <b>BLINFS01\Users </b>group the only domain user that's allowed to access the directory is me. But how? Those are local users only, not the domain ones. Anyway I decided to test it. I tried logging in into the website back, but unfortunately, this time I got the following error message in my browser:</div>
<div>
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-UXRQz46yICw/UaMHtx1yppI/AAAAAAAAtpg/UxuYaBqn-gQ/s1600/blog3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" src="http://1.bp.blogspot.com/-UXRQz46yICw/UaMHtx1yppI/AAAAAAAAtpg/UxuYaBqn-gQ/s1600/blog3.png" title="IIS 8 permissions issues" /></a></div>
<br /></div>
<div>
<br /></div>
<div>
Either I was allowing any domain users or I was blocking everyone. It looks like I faced an impasse.</div>
<div>
<br /></div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Solution</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
After half of a day of troubleshooting I have finally fixed the above issue by applying the solutions described in the following <a href="http://www.iis.net/learn/manage/configuring-security/application-pool-identities">article</a>. It occurred that prior to <b>IIS 7.5 </b>a new feature has been added that is called <b>AppPool Identities</b> that for each of the Application Pools there is a unique account in the system that holds all the required permissions to host the website.<br />
<br />
I removed the<b> </b>permissions for <b>BLINFS01\Users </b>group then and added the same permissions for <b>IIS AppPool\BLINFS01 </b>user instead as shown on the attached picture:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-ZyiI0WzPtPI/UaMH5mTusLI/AAAAAAAAtpo/SdJrCDhFohE/s1600/blog4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" src="http://2.bp.blogspot.com/-ZyiI0WzPtPI/UaMH5mTusLI/AAAAAAAAtpo/SdJrCDhFohE/s1600/blog4.png" title="IIS AppPool Identities" /></a></div>
<br />
After pressing <b>Check Names </b>button the account name has been converted as shown on the attached picture:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-zlXb35uIEmk/UaMIDWwx2mI/AAAAAAAAtpw/Wka21gn-z-U/s1600/blog5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" border="0" src="http://3.bp.blogspot.com/-zlXb35uIEmk/UaMIDWwx2mI/AAAAAAAAtpw/Wka21gn-z-U/s1600/blog5.png" title="IIS AppPool Identities" /></a></div>
<br />
After applying the above permissions I was able to successfully log in into my website and moreover, no other domain user was able to. I find IIS AppPool Identities and the permissions associated with them very useful however I regret that they are so purely documented. Hope that the above article will help some of the wandered sysadmins that are experiencing the same issues.</div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-47710824306144466622013-05-21T22:29:00.002-07:002013-06-18T22:59:10.362-07:00SVN email notifications - svn-mod-email<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<div style="text-align: justify;">
I have recently published a module to subversion software for sending email notifications about commits. It is distributed as a debian package and can be downloaded from <a href="https://code.google.com/p/svn-mod-email/downloads/detail?name=svn-mod-email_1.0_all.deb&can=2&q=#makechanges">Google Code</a> under terms of the GPL. My fascination when writing this post is even bigger as this is my first Debian package downloadable publicly. I encourage anyone for testing it and sharing any constructive feedback.</div>
<div style="text-align: justify;">
<br /></div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">About the package</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
The package consists of two Perl scripts:</div>
<div>
<ul>
<li style="text-align: justify;"><b>svnennotification </b>- used to enable / disable email notifications for SVN repositories</li>
<li style="text-align: justify;"><b>svnnotification </b>- sends email notifications after SVN commit to repository</li>
</ul>
<div style="text-align: justify;">
For each of the scripts you may refer into their corresponding man pages to display all the available options. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The <b>svnnotification </b>script is destined to be run from the <b>post-commit </b>hook and it shouldn't be executed manually. It provides an engine for gathering all the information about the commit and building a list of recipients that should be notified by email message. It also actualy sends an email notification to the recipients. The scripts uses <b>email </b>and <b>email.conf </b>files inside the <b>conf </b>directory of the repository that are created by <b>svnennotification</b> script.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
While the <b>svnnotification </b>script is what actually does the job, the main purpose of the <b>svnennotification </b>script is to manage the <b>post-commit </b>hook of the repository and the configuration files inside its <b>conf </b>directory:</div>
<div style="text-align: justify;">
<ul>
<li><b>email </b>- determines SVN repositories, its sub-repositories and corresponding email addresses of users, that should receive email notifications</li>
<li><b>email.conf </b>- contains global settings regarding email system</li>
</ul>
</div>
<div style="text-align: justify;">
In general its purpose is to either enable or disable email notification for the SVN repository.</div>
</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
As for the scripts you may also refer into man pages of the configuration files for all the detailed information about their structure.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<h3 style="text-align: center;">
<span style="color: #b45f06;">Examples</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
The following section shows common example of package usage starting with its installation, going threw its configuration and finally ending up with working notifications.<br />
<b><br /></b>
<b>1) Installation:</b><br />
<br />
<div>
<br /></div>
<div>
To install the package first download it from the following <a href="http://svn-mod-email.googlecode.com/files/svn-mod-email_1.0_all.deb">website</a>:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># wget http://goo.gl/7i9ev</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">Then install the package as a regular Debian archive:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># dpkg -i svn-mod-email_1.0_all.deb</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">If you'll notice any error messages regarding missing dependencies, install them by running the following command:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># apt-get -f install</span></div>
<div>
<b><br /></b></div>
<div>
<b><span style="font-family: inherit;">2) </span>Configuration:</b></div>
<div>
<b><br /></b></div>
<div>
You're ready to start using the module. Now you need to run the <b>svnennotification </b>command to enable the notifications for the SVN repository. Simply run the command with a path to the repository as an argument:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"># svnennotification /srv/svn/repo</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">You may pass much more options into the command. Please, refer to the man pages for details. </span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">The above will create or update <b>/srv/svn/repo/hooks/post-commit </b>hook and create the <b>/srv/svn/repo/conf/email </b>and <b>/srv/svn/repo/conf/email.conf </b>files. Now you need to configure them. Lets start with the <b>email.conf </b>file that contains global settings regarding your email system. You may either refer to the man page for the file or simply read it that there is clearly explained what does any option stand for.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">After updating the <b>email.conf </b>file it's time to take care of the <b>email </b>file that actually determines SVN repository, </span>its sub-repositories and corresponding email addresses of users that should receive email notifications. The structure of the file is similar to the <b>authz </b>file, but instead of access rights it determines email addresses of the recipients. The following is a simple example of the file:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">[repo:/]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">user1@domain.com</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">[repo:/subrepo]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">user1@domain.com</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">user2@domain.com</span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: inherit;">In such a case if a commit is made to the main repository only user1 will receive a notification. Whether the commit is made to the subrepo directory inside the main repo anyway both user1 and user2 will receive email notification. Please, refer to the man page for the file to read more about its contect and configuration possibilities.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;"><b>3) Usage:</b></span></div>
<div>
<span style="font-family: inherit;"><b><br /></b></span></div>
<div>
<span style="font-family: inherit;">After performing all the above steps there are no more requirements to start using the module. Simply run your first commit into the repository and enjoy email notifications about the changes that you've just made!</span></div>
</div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-81830889334358327352013-05-16T02:12:00.000-07:002013-05-17T02:14:34.621-07:00ProFTPD - logging to MySQL database<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<div style="text-align: justify;">
ProFTPD is a powerfull and versatile FTP server however because of its rare usage, or even because of rare usage of FTP nowadays, it suffers from lack of the support as for the HowTos, forum topics, etc. One of the greatest features of the ProFTPD is the capability of storing logs in the database instead of files that is provided by its built-in mechanisms. My goal was to store all the read/write attempts. I created custom log for that purpose that is then being parsed by the ProFTPD MySQL module to extract the following information:</div>
<div style="text-align: justify;">
</div>
<ul>
<li><b>who </b>attempted to access the FTP server (username + IP)</li>
<li><b>when </b>did the attempt take place</li>
<li><b>what </b>exact operation was attempted</li>
</ul>
<div>
However ProFTPD supports many different backends I used the MySQL with default settings.</div>
<div>
<br /></div>
<div>
<div style="text-align: justify;">
<span style="color: orange; text-align: left;"><b>Note</b></span><span style="text-align: left;">: </span>The following HowTo assumes that you have both ProFTPD and MySQL servers installed and configured. <span style="text-align: left;">The MySQL server doesn't necessarily need to be installed on the same box as the ProFTPD, but if so it needs to be accessible over the network. In my case both ProFTPD and MySQL were installed on the same box that was running Ubuntu Server 12.04 amd64.</span></div>
</div>
<div>
<div style="text-align: justify;">
<br /></div>
</div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Configuration</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
1) Install the MySQL module for ProFTPD:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">apt-get install proftpd-mod-mysql</span></div>
<div>
<br /></div>
<div>
<div style="text-align: justify;">
2) Add the following line into the main ProFTPD configuration file (<b>/etc/proftpd/proftpd.conf</b>) to include the MySQL module configuration file:</div>
</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Include /etc/proftpd/sql.conf</span></div>
<div>
<br /></div>
<div>
<div style="text-align: justify;">
3) Add the following lines into the main ProFTPD configuration file to create and define the extended log file that will track all the read / write attempts:</div>
<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">ExtendedLog /var/log/proftpd/ftp_access.log WRITE,READ write</span><br />
<span style="font-family: Courier New, Courier, monospace;">LogFormat write "%h %l %u %t \"%r\" %s %b"</span><br />
<br />
<br />
<div style="text-align: justify;">
4) Make sure that the following lines are not commented out inside the ProFTPD modules configuration file (<b>/etc/proftpd/modules.conf</b>):</div>
</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">LoadModule mod_sql.c</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">LoadModule mod_sql_mysql.c</span></div>
<div>
<br /></div>
<div>
<div style="text-align: justify;">
5) Edit the MySQL module configuration file (<b>/etc/proftpd/sql.conf</b>) that it looks like on the example below:</div>
</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><IfModule mod_sql.c></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>SQLBackend mysql</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>SQLEngine on</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>SQLLogFile /var/log/proftpd/sql.log</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>SQLConnectInfo [database]@[host] [username] [password]</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>SQLLogFile /var/log/proftpd/ftp_access.log</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span><span style="white-space: pre;">SQLLog DELE,MKD,RETR,RMD,RNFR,RNTO,STOR,APPE extendedlog</span></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span><span style="white-space: pre;">SQLNamedQuery extendedlog FREEFORM "INSERT INTO proftpd_access_log (`IP`, `username`, `time`, `operation`) VALUES ('%a', '%u', NOW(), '%r')"</span></span></div>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"></IfModule></span></div>
<div>
<br /></div>
<div style="text-align: justify;">
First part of the configuration should stay as it is.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Inside the <b>SQLConnectInfo</b> directive you should provide your MySQL server details as for the database name, IP address or FQDN of the MySQL server and finally username and password that has RW access into the database. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The third part of the configuration is responsible for extracting and parsing the content of ProFTPD access log file and putting the values into the database. Unless you don't want to change the expected content of the log described in the Intro section, you shouldn't change this configuration.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
6) Create the table in MySQL database for log information storing purpose:<br />
<span style="font-family: inherit;"><br /></span>
<br />
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">CREATE TABLE proftpd_access_log (id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT, IP VARCHAR(50) NOT NULL, username VARCHAR(50), time DATETIME NOT NULL, operation VARCHAR(500) NOT NULL, PRIMARY KEY (id)) COLLATE=utf8_general_ci ENGINE=MyISAM;</span></div>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">7) Finally, restart the ProFTPD server:</span></div>
<div style="text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;">/etc/init.d/proftpd restart</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: inherit;">You'll notice your <b>proftpd_access_log </b>table keeps filling in:</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">mysql> select * from proftpd_access_log;</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+----+--------------+------------------+---------------------+---------------+</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| id | IP | username | time | operation |</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+----+--------------+------------------+---------------------+---------------+</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">| 1 | 1.2.3.4 | someuser | 2013-05-16 10:48:49 | RETR conf.txt |</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">+----+--------------+------------------+---------------------+---------------+</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">1 row in set (0.01 sec)</span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
</div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">References</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div style="text-align: justify;">
The above HowTo shows only one particular example of using ProFTPD built-in mechanisms of storing logs in the database. The available amount of possibilities and information to store is almost infinit. Please, refer to the following articles for more information:</div>
<div style="text-align: justify;">
<ul>
<li><a href="http://www.proftpd.org/docs/howto/FTP.html">FTP commands</a></li>
<li><a href="http://www.proftpd.org/docs/directives/linked/config_ref_ExtendedLog.html">ExtendedLog directive</a></li>
<li><a href="http://www.proftpd.org/docs/directives/linked/config_ref_LogFormat.html">LogFormat directive</a></li>
<li><a href="http://www.proftpd.org/docs/directives/linked/config_ref_SQLLog.html">SQLLog directive</a></li>
<li><a href="http://www.proftpd.org/docs/directives/linked/config_ref_SQLNamedQuery.html">SQLNamedQuery</a></li>
</ul>
</div>
<br /></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-31839274711052794012013-05-10T23:31:00.001-07:002013-05-10T23:31:15.800-07:00Could not perform immediate configuration on 'util-linux' - issues during the upgrade from Ubuntu 10.04 to 12.04 (part 3)<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;"><br class="Apple-interchange-newline" /><br class="Apple-interchange-newline" />Intro</span></h3>
<br />
<div style="text-align: justify;">
When performing an upgrade of Ubuntu <b>10.04 </b>to <b>12.04 </b>I experienced some serious issues with the <b>util-linux</b> package. What's interesting is that I haven't found any official website for this bug. Thanks to some different tips found in the I have managed to bypass those issues anyway. The following steps describe a detailed troubleshooting process both with a solution.<br />
<br />
<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;">Symptoms</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<ul>
<li>During the regular upgrade process, when running the following command:</li>
</ul>
<div>
<b><span style="font-family: Courier New, Courier, monospace;">apt-get dist-upgrad</span><span style="font-family: Courier New, Courier, monospace;">e</span></b></div>
<div>
<b><span style="font-size: x-small;"><span style="font-family: Courier New, Courier, monospace;"><br /></span></span></b></div>
<div>
<span style="font-family: inherit;">the following error message is being displayed:</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b>E: Could not perform immediate configuration on 'util-linux'.Please see man 5 apt.conf under APT::Immediate-Configure for details. (2)</b></span></div>
</div>
<div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><b><br /></b></span></div>
</div>
<div>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Solution</span></h3>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
Please, run the following sequence of commands:</div>
<div>
<ul>
<li><b><span style="font-family: Courier New, Courier, monospace;">apt-get install libc6-dev</span></b></li>
<li><b style="font-family: 'Courier New', Courier, monospace;">apt-get install util-linux -f -o APT::Immediate-Configure=0</b></li>
<li><span style="font-family: Courier New, Courier, monospace;"><b>apt-get dist-upgrade</b></span></li>
<li><span style="font-family: inherit;">follow the regular on-screen instructions to accomplish the upgrade process</span></li>
<li><b style="font-family: 'Courier New', Courier, monospace;">apt-get autoremove</b></li>
<li><b style="font-family: 'Courier New', Courier, monospace; line-height: 18px; text-align: left;">reboot</b></li>
</ul>
<div style="text-align: left;">
<h3 style="text-align: center;">
<span style="color: #b45f06;"><br /></span></h3>
<h3 style="text-align: center;">
<span style="color: #b45f06;">Notes</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<div style="text-align: justify;">
Issues described in the above posts are not the only ones that I encountered when upgrading my <b>Ubuntu 10.04 </b>servers to <b>12.04</b>. Please, take a look on the following post descirbing similar issues both with the troubleshooting process and the solution:</div>
</div>
<div>
<br /></div>
<div>
<a href="http://tkurek.blogspot.com/2013/04/issues-with-python-minimal-during.html">http://tkurek.blogspot.com/2013/04/issues-with-python-minimal-during.html</a><br />
<a href="http://tkurek.blogspot.com/2013/04/could-not-perform-immediate.html">http://tkurek.blogspot.com/2013/04/could-not-perform-immediate.html</a></div>
</div>
</div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com0tag:blogger.com,1999:blog-4818704452869317626.post-79092976440277380752013-05-08T00:51:00.002-07:002013-05-08T07:23:15.684-07:00VMFS: Unsupported version 5 - How to mount VMFS5 on Ubuntu<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;">Intro</span></h3>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The VMFS is great, but it lacks from the support on Ubuntu and other Linux distributions. We all know the <b>vmfs-tools </b>package and the <b>vmfs-fuse </b>tool and there are a lot of great HowTos regarding its usage however its current official version - <b>0.2.1-1 </b>- doesn't support <b>VMFS5</b>. The following is an output from my terminal when trying to mount VMFS5 formatted disk (<b>/dev/sda1 </b>in my case):<br />
<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">root@ubuntu:~# vmfs-fuse /dev/sda1 /mnt/</span><br />
<span style="font-family: Courier New, Courier, monospace;">VMFS: Unsupported version 5</span><br />
<span style="font-family: Courier New, Courier, monospace;">Unable to open device/file "/dev/sda1".</span><br />
<br />
<br />
According to some various tips I have finally managed to mount the VMFS5 drive on Ubuntu using the manually compiled <b>0.2.5-1</b> version anyway, but the support is extremely limited so far (e.g. there's no <b>RW </b>support). What I found is that the goal of the developers is to introduce stable internal API and the <b>mkfs.vmfs </b>tool prior to version <b>0.5 </b>and the full write support prior to version <b>1.0</b>. So far, please find the following instruction that will enable you to mount your VMFS5 drives on Ubuntu in <b>RO </b>mode. I was using regural <b>Ubuntu Desktop 12.04 </b>Live CD.<br />
<br />
<br />
<h3 style="text-align: center;">
<span style="color: #b45f06;">Installation</span></h3>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
1) First of all boot the system from Ubuntu Desktop 12.04 Live CD and log in as a root user. Then download and extract <a href="https://launchpad.net/ubuntu/+archive/primary/+files/vmfs-tools_0.2.5-1.tar.gz">the package file</a>:</div>
<div>
<br /></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">root@ubuntu:~# cd /tmp</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">root@ubuntu:/tmp# wget https://launchpad.net/ubuntu/+archive/primary/+files/vmfs-tools_0.2.5-1.tar.gz</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">root@ubuntu:/tmp# tar -xzf vmfs-tools_0.2.5-1.tar.gz</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">root@ubuntu:/tmp/vmfs-tools# cd vmfs-tools</span></div>
<div>
<br /></div>
<div>
2) Install all the prerequisites. For best practice refer to the <b>README </b>file. In general the following should be enough:</div>
<div>
<br /></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">root@ubuntu:/tmp/vmfs-tools# apt-get install gcc make uuid-dev pkg-config libfuse-dev</span></div>
<div>
<br /></div>
<div>
If you need the man pages, you'll need to install <b>asciidoc</b>, <b>xsltproc </b>and <b>docbook-xsl </b>packages too.</div>
<div>
<br /></div>
<div>
3) Configure and compile the package:</div>
<div>
<br /></div>
<div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">root@ubuntu:/tmp/vmfs-tools# <b>./configure </b></span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">Checking for pkg-config...yes</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">Checking for uuid...yes</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">Checking for fuse...yes</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">Checking for asciidoc...no</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">Checking for xsltproc...no</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">Checking for docbook.xsl...no</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">Checking for strndup...yes</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">Checking for dlopen in -ldl...yes</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">Checking for posix_memalign...yes</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">echo "#if 1" > version</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">echo "#define VERSION \"v0.0.0.0.\"" >> version</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">echo "#else" >> version</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">echo VERSION := v0.0.0.0. >> version</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">echo "#endif" >> version</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">make: `config.cache' is up to date.</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">root@ubuntu:/tmp/vmfs-tools# <b>make</b></span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Idebugvmfs -Ilibvmfs -I/usr/include/uuid -Ilibreadcmd -include version -c -o debugvmfs/debugvmfs.o debugvmfs/debugvmfs.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Idebugvmfs -Ilibvmfs -I/usr/include/uuid -Ilibreadcmd -c -o debugvmfs/variables.o debugvmfs/variables.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/scsi.o libvmfs/scsi.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/utils.o libvmfs/utils.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/vmfs_bitmap.o libvmfs/vmfs_bitmap.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/vmfs_block.o libvmfs/vmfs_block.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/vmfs_dirent.o libvmfs/vmfs_dirent.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/vmfs_file.o libvmfs/vmfs_file.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/vmfs_fs.o libvmfs/vmfs_fs.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/vmfs_heartbeat.o libvmfs/vmfs_heartbeat.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/vmfs_host.o libvmfs/vmfs_host.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/vmfs_inode.o libvmfs/vmfs_inode.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/vmfs_lvm.o libvmfs/vmfs_lvm.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/vmfs_metadata.o libvmfs/vmfs_metadata.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibvmfs -I/usr/include/uuid -c -o libvmfs/vmfs_volume.o libvmfs/vmfs_volume.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">ar -r libvmfs/libvmfs.a libvmfs/scsi.o libvmfs/utils.o libvmfs/vmfs_bitmap.o libvmfs/vmfs_block.o libvmfs/vmfs_dirent.o libvmfs/vmfs_file.o libvmfs/vmfs_fs.o libvmfs/vmfs_heartbeat.o libvmfs/vmfs_host.o libvmfs/vmfs_inode.o libvmfs/vmfs_lvm.o libvmfs/vmfs_metadata.o libvmfs/vmfs_volume.o</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">ar: creating libvmfs/libvmfs.a</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">ranlib libvmfs/libvmfs.a</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ilibreadcmd -c -o libreadcmd/readcmd.o libreadcmd/readcmd.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">ar -r libreadcmd/libreadcmd.a libreadcmd/readcmd.o</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">ar: creating libreadcmd/libreadcmd.a</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">ranlib libreadcmd/libreadcmd.a</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -o debugvmfs/debugvmfs debugvmfs/debugvmfs.o debugvmfs/variables.o libvmfs/libvmfs.a libreadcmd/libreadcmd.a -ldl -luuid </span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ifsck.vmfs -Ilibvmfs -I/usr/include/uuid -include version -c -o fsck.vmfs/vmfs_fsck.o fsck.vmfs/vmfs_fsck.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -o fsck.vmfs/fsck.vmfs fsck.vmfs/vmfs_fsck.o libvmfs/libvmfs.a -luuid</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Iimager -c -o imager/imager.o imager/imager.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">imager/imager.c: In function ‘skip_zero_blocks’:</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">imager/imager.c:165:13: warning: ignoring return value of ‘ftruncate’, declared with attribute warn_unused_result [-Wunused-result]</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -o imager/imager imager/imager.o </span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ivmfs-fuse -Ilibvmfs -I/usr/include/uuid -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse -c -o vmfs-fuse/vmfs-fuse.o vmfs-fuse/vmfs-fuse.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -o vmfs-fuse/vmfs-fuse vmfs-fuse/vmfs-fuse.o libvmfs/libvmfs.a -luuid -pthread -lfuse -lrt -ldl</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -Wall -O2 -g -D_FILE_OFFSET_BITS=64 -Ivmfs-lvm -Ilibvmfs -I/usr/include/uuid -Ilibreadcmd -include version -c -o vmfs-lvm/vmfs-lvm.o vmfs-lvm/vmfs-lvm.c</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">gcc -o vmfs-lvm/vmfs-lvm vmfs-lvm/vmfs-lvm.o libvmfs/libvmfs.a libreadcmd/libreadcmd.a -ldl -luuid </span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">(echo "*.tar.gz"; \</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> echo "*.[ao]"; \</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> echo "*.xml"; \</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> echo "*.8"; \</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> echo "version"; \</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> echo "config.cache"; \</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> echo debugvmfs/debugvmfs; echo fsck.vmfs/fsck.vmfs; echo imager/imager; echo vmfs-fuse/vmfs-fuse; echo vmfs-lvm/vmfs-lvm; \</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>) > .gitignore</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">root@ubuntu:/tmp/vmfs-tools# <b>make install</b></span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">install -s -m 0755 debugvmfs/debugvmfs /usr/local/sbin/</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">install -s -m 0755 fsck.vmfs/fsck.vmfs /usr/local/sbin/</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">install -s -m 0755 vmfs-fuse/vmfs-fuse /usr/local/sbin/</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">install -s -m 0755 vmfs-lvm/vmfs-lvm /usr/local/sbin/</span></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">make: *** No rule to make target `debugvmfs/debugvmfs.8', needed by `/usr/local/share/man/man8/debugvmfs.8'. Stop.</span></div>
</div>
<div>
<br /></div>
<div>
4) Now you should be able to mount the <b>VMFS5</b> disk in <b>RO </b>mode without any issues:</div>
<div>
<br /></div>
<div style="text-align: left;">
<span style="font-family: Courier New, Courier, monospace;">root@ubuntu:/tmp/vmfs-tools# <b>/usr/local/sbin/vmfs-fuse /dev/sda1 /mnt</b></span></div>
<div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
root@ubuntu:/tmp/vmfs-tools# <b>mount</b></div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
/cow on / type overlayfs (rw)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
proc on /proc type proc (rw,noexec,nosuid,nodev)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
udev on /dev type devtmpfs (rw,mode=0755)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
/dev/sr0 on /cdrom type iso9660 (ro,noatime)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
/dev/loop0 on /rofs type squashfs (ro,noatime)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
none on /sys/fs/fuse/connections type fusectl (rw)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
none on /sys/kernel/debug type debugfs (rw)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
none on /sys/kernel/security type securityfs (rw)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
tmpfs on /tmp type tmpfs (rw,nosuid,nodev)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
none on /run/shm type tmpfs (rw,nosuid,nodev)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
gvfs-fuse-daemon on /home/ubuntu/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=ubuntu)</div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
<b>/dev/fuse on /mnt type fuse (rw,nosuid,nodev,default_permissions)</b></div>
<div style="font-family: 'Courier New', Courier, monospace; text-align: left;">
<b><br /></b></div>
<div style="text-align: left;">
<span style="font-family: inherit;">Don't be confused with the <b>rw</b> option. The disk is mounted in <b>RO </b>mode.</span></div>
</div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<span style="color: #b45f06;"><br /></span></div>
<div>
<span style="color: #b45f06;"><br /></span></div>
</div>
Tytus Kurekhttp://www.blogger.com/profile/09611273720793657156noreply@blogger.com10