Run Your Own - User contributions [en]

Wildcard Certificates with acme.sh

2025-08-17T15:05:45Z

320x200: /* Request a wildcard cert for lurk.org */

<code>acme.sh</code> is a lightweight shell script based tool to handle Let's Encrypt certificates, etc.

== Installation and maintenance ==
All commands are for <code>root</code>.
=== Install ===
wget https://get.acme.sh
sh acme.sh

This will install the script to <code>/root/.acme</code> and add it to path by sourcing a script from root's <code>.bashrc</code>

=== Maintenance ===
* upgrade
acme.sh --uprade
* force renew already configured/installed certs
acme.sh --renew-all --force
* what is wrong? (requires enabling logging in <code>account.conf</code>)
less /root/.acme.sh/acme.sh.log

== Request a wildcard cert for lurk.org ==
We use wildcard certificates with DNS authentification, and we use the DNS server of our registrar, porkbun. It's not great (terrible UI for DNS editing), but it's cheap. Porkbun DNS support was added in recent versions of <code>acme.sh</code>. To make it work, we first need to find our Porkbun API keys and use them to set the following environment variables in root's <code>.bashrc</code>:

export PORKBUN_API_KEY="..."
export PORKBUN_SECRET_API_KEY="..."

When ready and reloaded:

acme.sh --issue --dns dns_porkbun -d lurk.org -d *.lurk.org

Should be set and forget. Do not run the issue command again to force renew a cert, see maintenance section above.

result:

* cert is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.cer</code>
* cert key is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.key</code>
* intermediate CA cert is in: <code>/root/.acme.sh/lurk.org_ecc/ca.cer</code>
* full-chain cert is in: <code>/root/.acme.sh/lurk.org_ecc/fullchain.cer</code>

== Install the certs for nginx ==
The following command will install the certs for nginx, assuming there is a <code>/etc/nginx/certs/</code> directory. Should be set and forget. Do not run this command to force renew a cert, see maintenance section above.
mkdir /etc/nginx/certs
acme.sh --install-cert -d lurk.org -d *.lurk.org --key-file /etc/nginx/certs/key.pem --fullchain-file /etc/nginx/certs/cert.pem --reloadcmd "systemctl force-reload nginx"

== Deployment for other services ==
=== Configuration ===
<code>acme.sh</code> can also support custom installs of the certificates. They call this deployment, and all the scripts provided by the project can be found in <code>/root/.acme.sh/deploy</code>.

It's possible to make new deploy scripts quite easily, here is an example for <code>cooldaemon.sh</code>:
<pre>
# this makes accessible as variables all the necessary paths and files
cooldaemon_deploy() {
_cdomain="$1"
_ckey="$2"
_ccert="$3"
_cca="$4"
_cfullchain="$5"

_debug _cdomain "$_cdomain"
_debug _ckey "$_ckey"
_debug _ccert "$_ccert"
_debug _cca "$_cca"
_debug _cfullchain "$_cfullchain"

# make a var for the target location
_ssl_path="/etc/cooldaemon/certs/"

# cooldaemon only needs the fullchain perm and the key so
# we only copy these
cp $_ckey $_ssl_path
cp $_cfullchain $_ssl_path

# any extra commands can be added here for instance
# maybe cooldaemon is picky about cert ownership
chown -R cooldaemon:cooldaemon $_ssl_path

# last but not least we reload cool daemon
# please note that some other daemons may need a restart instead
systemctl reload mumble-server

return 0
}
</pre>

To enable the deployment at every cert renewal:
acme.sh --deploy -d lurk.org -d *.lurk.org --deploy-hook cooldaemon

=== Fine tuning ===
The configured hook can be found listed in the domain name conf file (for instance <code>lurk.org.conf</code>): <code>Le_DeployHook='cooldaemon,anothercooldaemon,'</code>.

== Old Stuff ==
Left for future ref in case we need this again in the future. Otherwise, please ignore and do not use

=== set up a cron job ===
24 3 * * * "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" --reloadcmd "systemctl force-reload nginx" --renew-hook "/root/icecast_certs.sh"

To be fully modern and cool, we should probably switch from cron to a systemd timer but that's for another day. There's also a deploy script for icecast so this could theoretically all be done in one run but there are no docs for --cron so it's not clear how to set this up. For now we use a post renewal hook that smooshes the certs together for icecast:

<pre>
#!/bin/sh
# turn the acme certs in to a certificate chain for icecast streaming
cat /root/.acme.sh/\*.lurk.org/fullchain.cer > /usr/local/share/icecast/icecast.pem
cat /root/.acme.sh/\*.lurk.org/\*.lurk.org.key >> /usr/local/share/icecast/icecast.pem
systemctl restart icecast

</pre>

[[Category:Certificates]]

Wildcard Certificates with acme.sh

2025-08-17T15:04:32Z

320x200: /* Install the certs for nginx */

<code>acme.sh</code> is a lightweight shell script based tool to handle Let's Encrypt certificates, etc.

== Installation and maintenance ==
All commands are for <code>root</code>.
=== Install ===
wget https://get.acme.sh
sh acme.sh

This will install the script to <code>/root/.acme</code> and add it to path by sourcing a script from root's <code>.bashrc</code>

=== Maintenance ===
* upgrade
acme.sh --uprade
* force renew already configured/installed certs
acme.sh --renew-all --force
* what is wrong? (requires enabling logging in <code>account.conf</code>)
less /root/.acme.sh/acme.sh.log

== Request a wildcard cert for lurk.org ==
We use wildcard certificates with DNS authentification, and we use the DNS server of our registrar, porkbun. It's not great (terrible UI for DNS editing), but it's cheap. Porkbun DNS support was added in recent versions of <code>acme.sh</code>. To make it work, we first need to find our Porkbun API keys and use them to set the following environment variables in root's <code>.bashrc</code>:

export PORKBUN_API_KEY="..."
export PORKBUN_SECRET_API_KEY="..."

When ready and reloaded:

acme.sh --issue --dns dns_porkbun -d lurk.org -d *.lurk.org

result:

* cert is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.cer</code>
* cert key is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.key</code>
* intermediate CA cert is in: <code>/root/.acme.sh/lurk.org_ecc/ca.cer</code>
* full-chain cert is in: <code>/root/.acme.sh/lurk.org_ecc/fullchain.cer</code>

== Install the certs for nginx ==
The following command will install the certs for nginx, assuming there is a <code>/etc/nginx/certs/</code> directory. Should be set and forget. Do not run this command to force renew a cert, see maintenance section above.
mkdir /etc/nginx/certs
acme.sh --install-cert -d lurk.org -d *.lurk.org --key-file /etc/nginx/certs/key.pem --fullchain-file /etc/nginx/certs/cert.pem --reloadcmd "systemctl force-reload nginx"

== Deployment for other services ==
=== Configuration ===
<code>acme.sh</code> can also support custom installs of the certificates. They call this deployment, and all the scripts provided by the project can be found in <code>/root/.acme.sh/deploy</code>.

It's possible to make new deploy scripts quite easily, here is an example for <code>cooldaemon.sh</code>:
<pre>
# this makes accessible as variables all the necessary paths and files
cooldaemon_deploy() {
_cdomain="$1"
_ckey="$2"
_ccert="$3"
_cca="$4"
_cfullchain="$5"

_debug _cdomain "$_cdomain"
_debug _ckey "$_ckey"
_debug _ccert "$_ccert"
_debug _cca "$_cca"
_debug _cfullchain "$_cfullchain"

# make a var for the target location
_ssl_path="/etc/cooldaemon/certs/"

# cooldaemon only needs the fullchain perm and the key so
# we only copy these
cp $_ckey $_ssl_path
cp $_cfullchain $_ssl_path

# any extra commands can be added here for instance
# maybe cooldaemon is picky about cert ownership
chown -R cooldaemon:cooldaemon $_ssl_path

# last but not least we reload cool daemon
# please note that some other daemons may need a restart instead
systemctl reload mumble-server

return 0
}
</pre>

To enable the deployment at every cert renewal:
acme.sh --deploy -d lurk.org -d *.lurk.org --deploy-hook cooldaemon

=== Fine tuning ===
The configured hook can be found listed in the domain name conf file (for instance <code>lurk.org.conf</code>): <code>Le_DeployHook='cooldaemon,anothercooldaemon,'</code>.

== Old Stuff ==
Left for future ref in case we need this again in the future. Otherwise, please ignore and do not use

=== set up a cron job ===
24 3 * * * "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" --reloadcmd "systemctl force-reload nginx" --renew-hook "/root/icecast_certs.sh"

To be fully modern and cool, we should probably switch from cron to a systemd timer but that's for another day. There's also a deploy script for icecast so this could theoretically all be done in one run but there are no docs for --cron so it's not clear how to set this up. For now we use a post renewal hook that smooshes the certs together for icecast:

<pre>
#!/bin/sh
# turn the acme certs in to a certificate chain for icecast streaming
cat /root/.acme.sh/\*.lurk.org/fullchain.cer > /usr/local/share/icecast/icecast.pem
cat /root/.acme.sh/\*.lurk.org/\*.lurk.org.key >> /usr/local/share/icecast/icecast.pem
systemctl restart icecast

</pre>

[[Category:Certificates]]

Wildcard Certificates with acme.sh

2025-08-17T15:03:07Z

320x200: /* Deployment for other services */

<code>acme.sh</code> is a lightweight shell script based tool to handle Let's Encrypt certificates, etc.

== Installation and maintenance ==
All commands are for <code>root</code>.
=== Install ===
wget https://get.acme.sh
sh acme.sh

This will install the script to <code>/root/.acme</code> and add it to path by sourcing a script from root's <code>.bashrc</code>

=== Maintenance ===
* upgrade
acme.sh --uprade
* force renew already configured/installed certs
acme.sh --renew-all --force
* what is wrong? (requires enabling logging in <code>account.conf</code>)
less /root/.acme.sh/acme.sh.log

== Request a wildcard cert for lurk.org ==
We use wildcard certificates with DNS authentification, and we use the DNS server of our registrar, porkbun. It's not great (terrible UI for DNS editing), but it's cheap. Porkbun DNS support was added in recent versions of <code>acme.sh</code>. To make it work, we first need to find our Porkbun API keys and use them to set the following environment variables in root's <code>.bashrc</code>:

export PORKBUN_API_KEY="..."
export PORKBUN_SECRET_API_KEY="..."

When ready and reloaded:

acme.sh --issue --dns dns_porkbun -d lurk.org -d *.lurk.org

result:

* cert is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.cer</code>
* cert key is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.key</code>
* intermediate CA cert is in: <code>/root/.acme.sh/lurk.org_ecc/ca.cer</code>
* full-chain cert is in: <code>/root/.acme.sh/lurk.org_ecc/fullchain.cer</code>

== Install the certs for nginx ==
The following command will install the certs for nginx, assuming there is a <code>/etc/nginx/certs/</code> directory. Should be set and forget.
acme.sh --install-cert -d lurk.org -d *.lurk.org --key-file /etc/nginx/certs/key.pem --fullchain-file /etc/nginx/certs/cert.pem --reloadcmd "systemctl force-reload nginx"

== Deployment for other services ==
=== Configuration ===
<code>acme.sh</code> can also support custom installs of the certificates. They call this deployment, and all the scripts provided by the project can be found in <code>/root/.acme.sh/deploy</code>.

It's possible to make new deploy scripts quite easily, here is an example for <code>cooldaemon.sh</code>:
<pre>
# this makes accessible as variables all the necessary paths and files
cooldaemon_deploy() {
_cdomain="$1"
_ckey="$2"
_ccert="$3"
_cca="$4"
_cfullchain="$5"

_debug _cdomain "$_cdomain"
_debug _ckey "$_ckey"
_debug _ccert "$_ccert"
_debug _cca "$_cca"
_debug _cfullchain "$_cfullchain"

# make a var for the target location
_ssl_path="/etc/cooldaemon/certs/"

# cooldaemon only needs the fullchain perm and the key so
# we only copy these
cp $_ckey $_ssl_path
cp $_cfullchain $_ssl_path

# any extra commands can be added here for instance
# maybe cooldaemon is picky about cert ownership
chown -R cooldaemon:cooldaemon $_ssl_path

# last but not least we reload cool daemon
# please note that some other daemons may need a restart instead
systemctl reload mumble-server

return 0
}
</pre>

To enable the deployment at every cert renewal:
acme.sh --deploy -d lurk.org -d *.lurk.org --deploy-hook cooldaemon

=== Fine tuning ===
The configured hook can be found listed in the domain name conf file (for instance <code>lurk.org.conf</code>): <code>Le_DeployHook='cooldaemon,anothercooldaemon,'</code>.

== Old Stuff ==
Left for future ref in case we need this again in the future. Otherwise, please ignore and do not use

=== set up a cron job ===
24 3 * * * "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" --reloadcmd "systemctl force-reload nginx" --renew-hook "/root/icecast_certs.sh"

To be fully modern and cool, we should probably switch from cron to a systemd timer but that's for another day. There's also a deploy script for icecast so this could theoretically all be done in one run but there are no docs for --cron so it's not clear how to set this up. For now we use a post renewal hook that smooshes the certs together for icecast:

<pre>
#!/bin/sh
# turn the acme certs in to a certificate chain for icecast streaming
cat /root/.acme.sh/\*.lurk.org/fullchain.cer > /usr/local/share/icecast/icecast.pem
cat /root/.acme.sh/\*.lurk.org/\*.lurk.org.key >> /usr/local/share/icecast/icecast.pem
systemctl restart icecast

</pre>

[[Category:Certificates]]

Wildcard Certificates with acme.sh

2025-08-17T15:00:05Z

320x200: /* Old Stuff */

<code>acme.sh</code> is a lightweight shell script based tool to handle Let's Encrypt certificates, etc.

== Installation and maintenance ==
All commands are for <code>root</code>.
=== Install ===
wget https://get.acme.sh
sh acme.sh

This will install the script to <code>/root/.acme</code> and add it to path by sourcing a script from root's <code>.bashrc</code>

=== Maintenance ===
* upgrade
acme.sh --uprade
* force renew already configured/installed certs
acme.sh --renew-all --force
* what is wrong? (requires enabling logging in <code>account.conf</code>)
less /root/.acme.sh/acme.sh.log

== Request a wildcard cert for lurk.org ==
We use wildcard certificates with DNS authentification, and we use the DNS server of our registrar, porkbun. It's not great (terrible UI for DNS editing), but it's cheap. Porkbun DNS support was added in recent versions of <code>acme.sh</code>. To make it work, we first need to find our Porkbun API keys and use them to set the following environment variables in root's <code>.bashrc</code>:

export PORKBUN_API_KEY="..."
export PORKBUN_SECRET_API_KEY="..."

When ready and reloaded:

acme.sh --issue --dns dns_porkbun -d lurk.org -d *.lurk.org

result:

* cert is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.cer</code>
* cert key is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.key</code>
* intermediate CA cert is in: <code>/root/.acme.sh/lurk.org_ecc/ca.cer</code>
* full-chain cert is in: <code>/root/.acme.sh/lurk.org_ecc/fullchain.cer</code>

== Install the certs for nginx ==
The following command will install the certs for nginx, assuming there is a <code>/etc/nginx/certs/</code> directory. Should be set and forget.
acme.sh --install-cert -d lurk.org -d *.lurk.org --key-file /etc/nginx/certs/key.pem --fullchain-file /etc/nginx/certs/cert.pem --reloadcmd "systemctl force-reload nginx"

== Deployment for other services ==
<code>acme.sh</code> can also support custom installs of the certificates. They call this deployment, and all the scripts provided by the project can be found in <code>/root/.acme.sh/deploy</code>.

It's possible to make new deploy scripts quite easily, here is an example for <code>cooldaemon.sh</code>:
<pre>
# this makes accessible as variables all the necessary paths and files
cooldaemon_deploy() {
_cdomain="$1"
_ckey="$2"
_ccert="$3"
_cca="$4"
_cfullchain="$5"

_debug _cdomain "$_cdomain"
_debug _ckey "$_ckey"
_debug _ccert "$_ccert"
_debug _cca "$_cca"
_debug _cfullchain "$_cfullchain"

# make a var for the target location
_ssl_path="/etc/cooldaemon/certs/"

# cooldaemon only needs the fullchain perm and the key so
# we only copy these
cp $_ckey $_ssl_path
cp $_cfullchain $_ssl_path

# any extra commands can be added here for instance
# maybe cooldaemon is picky about cert ownership
chown -R cooldaemon:cooldaemon $_ssl_path

# last but not least we reload cool daemon
# please note that some other daemons may need a restart instead
systemctl reload mumble-server

return 0
}
</pre>

To enable the deployment at every cert renewal:
acme.sh --deploy -d lurk.org -d *.lurk.org --deploy-hook cooldaemon

== Old Stuff ==
Left for future ref in case we need this again in the future. Otherwise, please ignore and do not use

=== set up a cron job ===
24 3 * * * "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" --reloadcmd "systemctl force-reload nginx" --renew-hook "/root/icecast_certs.sh"

To be fully modern and cool, we should probably switch from cron to a systemd timer but that's for another day. There's also a deploy script for icecast so this could theoretically all be done in one run but there are no docs for --cron so it's not clear how to set this up. For now we use a post renewal hook that smooshes the certs together for icecast:

<pre>
#!/bin/sh
# turn the acme certs in to a certificate chain for icecast streaming
cat /root/.acme.sh/\*.lurk.org/fullchain.cer > /usr/local/share/icecast/icecast.pem
cat /root/.acme.sh/\*.lurk.org/\*.lurk.org.key >> /usr/local/share/icecast/icecast.pem
systemctl restart icecast

</pre>

[[Category:Certificates]]

Wildcard Certificates with acme.sh

2025-08-17T14:58:17Z

320x200:

<code>acme.sh</code> is a lightweight shell script based tool to handle Let's Encrypt certificates, etc.

== Installation and maintenance ==
All commands are for <code>root</code>.
=== Install ===
wget https://get.acme.sh
sh acme.sh

This will install the script to <code>/root/.acme</code> and add it to path by sourcing a script from root's <code>.bashrc</code>

=== Maintenance ===
* upgrade
acme.sh --uprade
* force renew already configured/installed certs
acme.sh --renew-all --force
* what is wrong? (requires enabling logging in <code>account.conf</code>)
less /root/.acme.sh/acme.sh.log

== Request a wildcard cert for lurk.org ==
We use wildcard certificates with DNS authentification, and we use the DNS server of our registrar, porkbun. It's not great (terrible UI for DNS editing), but it's cheap. Porkbun DNS support was added in recent versions of <code>acme.sh</code>. To make it work, we first need to find our Porkbun API keys and use them to set the following environment variables in root's <code>.bashrc</code>:

export PORKBUN_API_KEY="..."
export PORKBUN_SECRET_API_KEY="..."

When ready and reloaded:

acme.sh --issue --dns dns_porkbun -d lurk.org -d *.lurk.org

result:

* cert is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.cer</code>
* cert key is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.key</code>
* intermediate CA cert is in: <code>/root/.acme.sh/lurk.org_ecc/ca.cer</code>
* full-chain cert is in: <code>/root/.acme.sh/lurk.org_ecc/fullchain.cer</code>

== Install the certs for nginx ==
The following command will install the certs for nginx, assuming there is a <code>/etc/nginx/certs/</code> directory. Should be set and forget.
acme.sh --install-cert -d lurk.org -d *.lurk.org --key-file /etc/nginx/certs/key.pem --fullchain-file /etc/nginx/certs/cert.pem --reloadcmd "systemctl force-reload nginx"

== Deployment for other services ==
<code>acme.sh</code> can also support custom installs of the certificates. They call this deployment, and all the scripts provided by the project can be found in <code>/root/.acme.sh/deploy</code>.

It's possible to make new deploy scripts quite easily, here is an example for <code>cooldaemon.sh</code>:
<pre>
# this makes accessible as variables all the necessary paths and files
cooldaemon_deploy() {
_cdomain="$1"
_ckey="$2"
_ccert="$3"
_cca="$4"
_cfullchain="$5"

_debug _cdomain "$_cdomain"
_debug _ckey "$_ckey"
_debug _ccert "$_ccert"
_debug _cca "$_cca"
_debug _cfullchain "$_cfullchain"

# make a var for the target location
_ssl_path="/etc/cooldaemon/certs/"

# cooldaemon only needs the fullchain perm and the key so
# we only copy these
cp $_ckey $_ssl_path
cp $_cfullchain $_ssl_path

# any extra commands can be added here for instance
# maybe cooldaemon is picky about cert ownership
chown -R cooldaemon:cooldaemon $_ssl_path

# last but not least we reload cool daemon
# please note that some other daemons may need a restart instead
systemctl reload mumble-server

return 0
}
</pre>

To enable the deployment at every cert renewal:
acme.sh --deploy -d lurk.org -d *.lurk.org --deploy-hook cooldaemon

== Old Stuff ==
Left for future ref in case we need this again in the future. Otherwise, please ignore and do not use

=== set up a cron job ===
24 3 * * * "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" --reloadcmd "systemctl force-reload nginx" --renew-hook "/root/icecast_certs.sh"

To be fully modern and cool, we should probably switch from cron to a systemd timer but that's for another day. There's also a deploy script for icecast so this could theoretically all be done in one run but there are no docs for --cron so it's not clear how to set this up. For now we use a post renewal hook that smooshes the certs together for icecast:

<pre>
#!/bin/sh
# turn the acme certs in to a certificate chain for icecast streaming
cat /root/.acme.sh/\*.lurk.org/fullchain.cer > /usr/local/share/icecast/icecast.pem
cat /root/.acme.sh/\*.lurk.org/\*.lurk.org.key >> /usr/local/share/icecast/icecast.pem
systemctl restart icecast

</pre>

[[Category:Certificates]]

Wildcard Certificates with acme.sh

2025-08-17T14:47:31Z

320x200: /* set up a cron job */

<code>acme.sh</code> is a lightweight shell script based tool to handle Let's Encrypt certificates, etc.

== Install the bash script ==
wget https://get.acme.sh

As root:

sh acme.sh

This will install the script to <code>/root/.acme</code> and add it to path by sourcing a script from root's <code>.bashrc</code>

== Request a wildcard cert for lurk.org ==
We use wildcard certificates with DNS authentification, and we use the DNS server of our registrar, porkbun. It's not great (terrible UI for DNS editing), but it's cheap. Porkbun DNS support was added in recent versions of <code>acme.sh</code>. To make it work, we first need to find our Porkbun API keys and use them to set the following environment variables in root's <code>.bashrc</code>:

export PORKBUN_API_KEY="..."
export PORKBUN_SECRET_API_KEY="..."

When ready and reloaded:

acme.sh --issue --dns dns_porkbun -d lurk.org -d *.lurk.org

result:

* cert is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.cer</code>
* cert key is in: <code>/root/.acme.sh/lurk.org_ecc/lurk.org.key</code>
* intermediate CA cert is in: <code>/root/.acme.sh/lurk.org_ecc/ca.cer</code>
* full-chain cert is in: <code>/root/.acme.sh/lurk.org_ecc/fullchain.cer</code>

== Install the certs for nginx ==
The following command will install the certs for nginx, assuming there is a <code>/etc/nginx/certs/</code> directory. Should be set and forget.
acme.sh --install-cert -d lurk.org -d *.lurk.org --key-file /etc/nginx/certs/key.pem --fullchain-file /etc/nginx/certs/cert.pem --reloadcmd "systemctl force-reload nginx"

== Deployment for other services ==
<code>acme.sh</code> can also support custom installs of the certificates. They call this deployment, and all the scripts provided by the project can be found in <code>/root/.acme.sh/deploy</code>.

It's possible to make new deploy scripts quite easily, here is an example for <code>cooldaemon.sh</code>:
<pre>
# this makes accessible as variables all the necessary paths and files
cooldaemon_deploy() {
_cdomain="$1"
_ckey="$2"
_ccert="$3"
_cca="$4"
_cfullchain="$5"

_debug _cdomain "$_cdomain"
_debug _ckey "$_ckey"
_debug _ccert "$_ccert"
_debug _cca "$_cca"
_debug _cfullchain "$_cfullchain"

# make a var for the target location
_ssl_path="/etc/cooldaemon/certs/"

# cooldaemon only needs the fullchain perm and the key so
# we only copy these
cp $_ckey $_ssl_path
cp $_cfullchain $_ssl_path

# any extra commands can be added here for instance
# maybe cooldaemon is picky about cert ownership
chown -R cooldaemon:cooldaemon $_ssl_path

# last but not least we reload cool daemon
# please note that some other daemons may need a restart instead
systemctl reload mumble-server

return 0
}
</pre>

To enable the deployment at every cert renewal:
acme.sh --deploy -d lurk.org -d *.lurk.org --deploy-hook cooldaemon

== Old Stuff ==
Left for future ref in case we need this again in the future. Otherwise, please ignore and do not use

=== set up a cron job ===
24 3 * * * "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" --reloadcmd "systemctl force-reload nginx" --renew-hook "/root/icecast_certs.sh"

To be fully modern and cool, we should probably switch from cron to a systemd timer but that's for another day. There's also a deploy script for icecast so this could theoretically all be done in one run but there are no docs for --cron so it's not clear how to set this up. For now we use a post renewal hook that smooshes the certs together for icecast:

<pre>
#!/bin/sh
# turn the acme certs in to a certificate chain for icecast streaming
cat /root/.acme.sh/\*.lurk.org/fullchain.cer > /usr/local/share/icecast/icecast.pem
cat /root/.acme.sh/\*.lurk.org/\*.lurk.org.key >> /usr/local/share/icecast/icecast.pem
systemctl restart icecast

</pre>

[[Category:Certificates]]

Simple LAN filesharing with WebDAV

2024-12-30T12:33:59Z

320x200: /* Linux filesystem */

'''WebDAV''' is both a quite popular and yet overlooked way to access and edit files remotely across a wide range of operating systems. Yes it's web stuff, again, but surprisingly fast, lightweight, and that can recover quite well on unstable networks or when the server has to be restarted, or has gone for lunch. A reason why it may be overlooked is possibly because it's often associated with sausage factories like own/nextcloud, or standalone implementations that are not particularly exciting. What is less known is that many web servers come with their own '''WebDAV''' implementation. Out of the usual suspects, '''nginx''', '''Apache''', and '''lighttpd''', the latter has both the most lightweight and most complete implementation. No need for anything else!

In these notes we only cover a simple LAN setup, you can build upon it for more complex use case of course.

== Server side ==
=== Installation ===
* This is for Debian, but I know you're smart, you will figure it out for <code>${FLAVOUR_OF_THE_MONTH_DISTRO}</code>
sudo apt install lighttpd lighttpd-mod-webdav

=== Example Configuration ===
Basically the configuration that matters for now are in <code>/etc/lighttpd/conf-available</code> and with symlinks in <code>/etc/lighttpd/conf-enabled</code>. '''In our simple example we will have three shared folders, one that can be mounted read-only by anyone, and two that are read-write but require a username and password'''.

* By default a temp config file called <code>99-unconfigured.conf</code> provides a generic landing page. We don't need it and we just have to enable the authentication config.
sudo lighttpd-disable-mod unconfigured
sudo lighttpd-enable-mod auth
* Create a user and password for the read-write share
sudo apt install apache2-utils
sudo htpasswd -c /etc/lighttpd/user.htpasswd turtleprincess heygirl
* create a new configuration file <code>/etc/lighttpd/conf-available/66-webdav.conf</code> with the following:
<pre>
server.modules += ( "mod_webdav" )
dir-listing.encoding = "utf-8"

# This is needed for keepings tracks of locks and props which
# are needed for shares that can be edited
webdav.sqlite-db-name = "/var/cache/lighttpd/lighttpd.webdav.db"

# auth
server.modules += ("mod_authn_file")
auth.backend = "htpasswd"
auth.backend.htpasswd.userfile = "/etc/lighttpd/user.htpasswd"
auth.require = ( "/readwrite1" => ( "method" => "basic",
"realm" => "YOU WOT MATE",
"require" => "valid-user" ),
"/readwrite2" => ( "method" => "basic",
"realm" => "YOU WOT MATE",
"require" => "valid-user" ))

# read-only stuff
$HTTP["url"] =~ "^/readonly(?:/|$)" {
alias.url = ( "/readonly" => "/local/path/to/readonly" )
dir-listing.activate = "enable"
webdav.activate = "enable"
webdav.is-readonly = "enable"
}

# shared pit of madness
# for users listed in /etc/lighttpd/user.htpasswd
$HTTP["url"] =~ "^/readwrite1(?:/|$)" {
alias.url = ( "/readwrite1" => "/local/path/to/readwrite1" )
dir-listing.activate = "enable"
webdav.activate = "enable"
webdav.is-readonly = "disable"
}
$HTTP["url"] =~ "^/readwrite2(?:/|$)" {
alias.url = ( "/readwrite2" => "/local/path/to/readwrite2" )
dir-listing.activate = "enable"
webdav.activate = "enable"
webdav.is-readonly = "disable"
}
</pre>

Now if you wonder what kind of magic will happen so that the lighttpd process, local and remote users can happily live together ever after, and edit the same files, well, none. It won't happen just like this, it does not work and you will be sad. Then you will start binge drinking to forget about server administration. You will go in the streets, pick up fights with strangers who are in a much better physical condition because they don't spend hours configuring neovim plugins that ''you haven't even used once''. It will be painful. So to avoid this unfortunate situation you need to do two things:

* First, the folder needs to be group-owned by the lighttpd process owner, in our case, on Debian, it's <code>www-data</code>. You also need to set the group ID bit on the folder you intend to use as WedDAV share, so that all newly created files will inherit the group ownership of this folder.
sudo chown regular_user:www-data /local/path/to/readwrite1
sudo chmod g+ws /local/path/to/readwrite1
* Second, you need to change the way lighttpd is started so that its umask is set in a way that any permission may be set for user and group (default is just user). On a systemd based system (haters gonna hate) you need to edit <code>/etc/systemd/system/lighttpd.service</code> like this:
ExecStart=/bin/sh -c 'umask 002;/usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf'

That's it, you have a working simple LAN filesharing with WebDAV check the documentation for more options, and more fun config time, at home, where it's much safer for you.

== Client side ==
=== Web browser ===
The nice advantage of the setup above is that all the files can be readily browsed at the local IP of the machine running lighttpd, for instance http://192.168.100.100/readonly. If you're into web stuff have a look at https://github.com/dom111/webdav-js, it would be really trivial to serve this js file with the lighttpd server and provide more functionality than just browsing and downloading.

=== Linux filesystem ===
Mounting a WebDAV on Linux is straightforward with <code>davfs2</code> and requires minimal configuration. <code>davfs2</code> is also able to recover quite well from disconnections and potential read/write fuckery (there is a daemon and a local cache, but you don't need to worry about this). If the server is dead, that you need to poweroff your machine, and you still have something mounted, it's a good idea to unmount things first to speed up shutdown. No it won't lock like a little NFS shit but there is a timeout and you're a busy person.
* To mount manually the read-only folder from above:
sudo apt install davfs2
sudo mount -t davfs http://192.168.100.100/readonly /mnt/readonly
* To remember username passwords you can create a personal <code>~/.davfs2/secrets</code> file with:
http://192.168.100.100/readwrite1 turtleprincess heygirl
http://192.168.100.100/readwrite2 turtleprincess heygirl
etc
* permissions matters
chmod 600 ~/.davfs2/secrets
* You can also have everything configured in your <code>/etc/fstab</code> and then mount everything as a regular user:
<pre>
# My cool LAN WebDAV thing
http://192.168.100.100/readonly /mnt/readonly/ davfs user,uid=username,noauto 0 0
http://192.168.100.100/readwrite1 /mnt/readonly/ davfs user,uid=username,noauto 0 0
http://192.168.100.100/readwrite2 /mnt/readonly/ davfs user,uid=username,noauto 0 0
</pre>

=== FreeBSD filesystem ===
On FreeBSD it's basically the same as on Linux. lol. Of course not! What did you think? That would be too easy. You need to work hard to earn your condescending smug BSD coolness. At time of writing, <code>davfs2</code> is being [https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=267518 ported to FreeBSD by its author]. Don't hold your breath, but hopefully this will happen because the FreeBSD usual way to mount WebDAV stuff with <code>mount.webdavfs</code> just sucks ("Device not configured" on certain reading operations, not able to recover when the server disconnects). As a workaround, the all-you-can-eat-software-buffet <code>rclone</code> is the only reliable way to have WebDAV mounted in a FreeBSD filesystem.

* installation
pkg install rclone # yeah yeah you can use portmaster and disable stuff you will regret later
* load fuse
sudo kldload fusefs
* to make it permanent add the following to <code>/boot/loader.conf</code>
# Filesystems in Userspace
fusefs_load="YES
* create a <code>~/.config/rclone/rclone.conf</code> config file with:
<pre>
[192.168.100.100]
type = webdav
url = http://192.168.100.100/
vendor = other
user = turtleprincess
pass = SEE_BELOW
</pre>
* generate the password with this:
echo "heygirl" | rclone obscure -
* test if it works
sudo mkdir /mnt/readwrite2
sudo chown user:user /mnt/readwrite2
rclone mount 192.168.100.100:readwrite2 /mnt/readwrite2/ --vfs-cache-mode writes --daemon

You can also make entries in your <code>/etc/fstab</code>
* rclone can't really be used directly, so we will trick fuse
sudo ln -s /usr/local/bin/rclone /usr/local/bin/mount.rclone
* in <code>/etc/fstab</code>
<pre>
# My cool LAN WebDAV thing
192.168.100.100:readonly /mnt/readonly fuse noauto,ro,mountprog=/usr/local/bin/mount.rclone
192.168.100.100:readwrite1 /mnt/readwrite1 fuse noauto,rw,mountprog=/usr/local/bin/mount.rclone
192.168.100.100:readwrite1 /mnt/readwrite1 fuse noauto,rw,mountprog=/usr/local/bin/mount.rclone
</pre>

=== Nautilus ===
For those into GUI (not judging, it's a free country) WebDAV is really easy to access thanks to the GVfs daemon. It's also a solid option in terms of recovery and making sure little to no fuckery is possible when writing on an unstable network. In practice, click on the "other" location button on the left pane, choose WedDAV and enter the URL in the form of <code>dav://192.168.100.100/readwrite1</code> and give your credentials. The only caveat is that the mounted folder name will be the server name, only the server name (IP in this case), so you may want to add your share to the bookmarks, and rename the bookmark to something more meaningful.

=== Android ===
Several commercial, closed source options. We don't talk to these people. There are however three interesting FLOSS options:
* https://github.com/newhinton/Round-Sync - super complete, multiple network file storage supported (based on rclone)
* https://github.com/phpbg/easysync - minimal automated WebDAV sync for the usual Android user data folders
* https://github.com/bitfireAT/davx5-ose - focused on CalDAV/CardDAV but hidden in one menu there is a mini browsing option for WebDAV

=== Nintendo Switch ===
Because you know, reasons:
* https://github.com/cy33hc/switch-ezremote-client - simple file browser/editor
* https://github.com/J-D-K/JKSV - popular save file manager, a WebDAV folder can be the default folder to export save backups

== Pro Tips ==
* IPs are used above but you can use a local DNS or make use of your local <code>/etc/hosts</code> files to give a nice name to your WedDAV server, look into RFC 8375 for pointers.
* To debug things if things are not behaving as nice things:
lighttpd-enable-mod accesslog
systemctl restart lighttpd
tail -f /var/log/lighttpd/access.log
# 17.4 hours later
# Problem solved at last
lighttpd-disable-mod accesslog
systemctl restart lighttpd

[[Category: WebDAV]]

Wildcard Certificates with acme.sh

2024-10-12T23:06:31Z

320x200: /* Deployment for other services */

Wildcard Certificates with acme.sh

2024-10-12T22:55:42Z

320x200: /* Install the certs for nginx */

Wildcard Certificates with acme.sh

2024-10-12T22:39:17Z

320x200: /* Install the certs for nginx */

Wildcard Certificates with acme.sh

2024-10-12T22:34:01Z

320x200:

Let's Encrypt Wildcard Certificates

2024-10-12T22:05:11Z

320x200: 320x200 moved page Let's Encrypt Wildcard Certificates to Wildcard Certificates with acme.sh

#REDIRECT [[Wildcard Certificates with acme.sh]]

Wildcard Certificates with acme.sh

2024-10-12T22:05:11Z

320x200: 320x200 moved page Let's Encrypt Wildcard Certificates to Wildcard Certificates with acme.sh

Wildcard Certificates with acme.sh

2024-10-12T22:04:29Z

320x200:

Wildcard Certificates with acme.sh

2024-10-12T21:59:11Z

320x200:

== Using acme.sh ==

<code>acme.sh</code> is a lightweight shell script based tool to handle Let's Encrypt certificates.

=== Install the bash script ===

wget https://get.acme.sh

As root:

sh acme.sh

This will install the script to <code>/root/.acme</code> and add it to path by sourcing a script from root's <code>.bashrc</code>

=== Request a wildcard cert for lurk.org ===

We use wildcard certificates with DNS authentification
first find and export the gandi dns key:

export GANDI_LIVEDNS_KEY="fdmlfsdklmfdkmqsdfk"

Then request a wildcard cert. (the dns key is added to a config file automatically for future renewals)

acme.sh --issue --dns dns_gandi_livedns --nginx -d *.lurk.org

Find the certs in:

/root/.acme.sh/\*.lurk.org/

=== Install the certs for nginx ===

=== Deployment for other services ===

[[Category:Certificates]]

Streaming Service with Icecast

2024-07-17T22:03:44Z

320x200: /* TLS/SSL Support */

'''Note:''' we will be using the <code>icecast-kh</code> fork that contains [https://github.com/karlheyes/icecast-kh/blob/master/NEWS some extra stuff and features/fixes/improvements] that may eventually land in vanilla <code>icecast</code>.

== Installation for a Simple Setup ==
=== Software ===
'''Note:''' At time of writing, <code>icecast-kh</code> suffers from [https://github.com/karlheyes/icecast-kh/issues/260 a small compilation problem with OpenSSL].

* Install dependencies (Debian)
apt install libxslt1-dev libogg-dev libvorbis-dev libtheora-dev libcurl4-openssl-dev
* Get the sources
cd /usr/src
git clone https://github.com/karlheyes/icecast-kh
* Compile and install
cd icecast-kh
./configure --with-openssl
make
make install

=== Firewall ===
* Make sure you listen on 8000, adjust your <code>iptables</code>:
-A INPUT -p tcp -m tcp --dport 8000 -j ACCEPT

You can adjust to your liking, 8000 is the default for Icecast.

=== Basic Configuration ===
Simple setup with <code>icecast</code> accepting 4 sources, changing process ownership to <code>nobody:nogroup</code>, and running in a <code>chroot</code>.

* log files in chroot:
mkdir /usr/local/share/icecast/log
chown nobody:nogroup /usr/local/share/icecast/log
* <code>/usr/local/etc/icecast.xml</code>:
<pre>
<icecast>
<location>𓅣</location>
<admin>top.cool@c_est.super.deluxe</admin>

<limits>
<clients>64</clients>
<sources>4</sources>
<queue-size>524288</queue-size>
<client-timeout>30</client-timeout>
<header-timeout>15</header-timeout>
<source-timeout>10</source-timeout>
<burst-size>65535</burst-size>
</limits>

<authentication>
<source-password>hackme</source-password>
<relay-password>hackme</relay-password>
<admin-user>admin</admin-user>
<admin-password>hackme</admin-password>
</authentication>

<hostname>stream.domain.tld</hostname>

<listen-socket>
<port>8000</port>
</listen-socket>

<fileserve>1</fileserve>

<paths>
<basedir>/usr/local/share/icecast</basedir>
<logdir>/log</logdir>
<webroot>/web</webroot>
<adminroot>/admin</adminroot>
<alias source="/" dest="/index.html"/>
</paths>

<logging>
<accesslog>access.log</accesslog>
<errorlog>error.log</errorlog>
<loglevel>1</loglevel> 
<logsize>10000</logsize> 
</logging>

<security>
<chroot>1</chroot>
<changeowner>
<user>nobody</user>
<group>nogroup</group>
</changeowner>
</security>
</icecast>
</pre>

=== Service file and autostart (systemd) ===
* Create a <code>/etc/systemd/system/icecast.service</code> unit file:
<pre>
[Unit]
Description=Icecast
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/icecast -c /usr/local/etc/icecast.xml
ExecReload=/usr/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target
</pre>
* Enable the service on boot:
systemctl enable icecast
* Manage the service with
service icecast start
service icecast status
service icecast stop

== MOAR Configuration ==
With the previous section you will get something up and running, stable and all. It's a good starting point to tweak things further.

=== NGINX Reverse Proxy ===
At time of writing, reverse proxying for icecast is not really worth it. Will explain a bit more eventually.

=== TLS/SSL Support ===
It's possible to use existing X.509 certificates to provide HTTPS access to both listeners and sources. For this to work the server and intermediate certificates with the private key.
* merge <code>fullchain.pem</code> with <code>privkey.pem</code>
# official acme client
cat /etc/letsencrypt/live/domain.tld/fullchain.pem > /usr/local/share/icecast/icecast.pem
cat /etc/letsencrypt/live/domain.tld/privkey.pem >> /usr/local/share/icecast/icecast.pem
# acme.sh
cat /root/.acme.sh/domain.tld/fullchain.cer > /usr/local/share/icecast/icecast.pem
cat /root/.acme.sh/domain.tld/domain.tld.key >> /usr/local/share/icecast/icecast.pem
* Adjust <code>icecast.xml</code> config file with <code><ssl-certificate></code> and <code><ssl-allowed-ciphers></code>:
<pre>
<paths>
<basedir>/usr/local/share/icecast</basedir>
<logdir>/log</logdir>
<webroot>/web</webroot>
<adminroot>/admin</adminroot>
<ssl-certificate>/usr/local/share/icecast/icecast.pem</ssl-certificate>
<ssl-allowed-ciphers>ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM: RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS</ssl-allowed-ciphers>
<alias source="/" dest="/index.html"/>
</paths>
</pre>

'''Note:''' If you have used the config file of this documentation, then icecast will be reachable both on <code>http://stream.domain.tld:8000</code> and <code>https://stream.domain.tld:port</code>. You may want to keep it like that as some clients (both for listening and emitting) do not support TLS/SSL. If you want enforce TLS/SSL, you can adjust your config file so that:
<pre>
<listen-socket>
<port>999</port>
<ssl>1</ssl>
</listen-socket>
</pre>

=== Moar Cores! ===
In <code>icecast-kh</code> it's possible to choose how many threads to use for processing clients. This should be based on the number of CPUs or cores. This can be adjusted in <code>icecast.xml</code> with the following line inside <code><limits></code>:
<workers>4</workers>

=== Relaying an External Stream (the lazy way) ===
This can be done simply by just pointing to the stream you want to relay. Careful though, you need to point to the remote server IP, not the domain. If you add <code><on-demand></code>, the stream will be relayed only if it is requested.
<pre>
<relay>
<server>10.10.10.10</server>
<port>1234</port>
<mount>/blabla.mp3</mount>
<local-mount>/blabla.mp3</local-mount>
<on-demand>1</on-demand>
<relay-shoutcast-metadata>0</relay-shoutcast-metadata>
</relay>
</pre>

[[Category: Streaming]]

Minimal TUI Wi-Fi Manager

2024-06-16T18:48:47Z

320x200: /* tweaks */

Good news! If you don't want to install third party daemons, gooey GUI things, or bloat just so you can connect to a wireless access point, then don't worry, Linux has all the CLI tools necessary to create a minimal on-demand TUI shell script thing to connect to, and manage these very Wi-Fi connections. The following info is for Debian, however the approach will be the same for other distros, including BSD operating systems that will differ mostly for some of the network commands. Why would you do this? I dunno, aesthetics, religious beliefs, limited resources, or just an opportunity to better understand what's running in the background.

== How does it work? ==
=== The things that mater ===

* <code>/etc/network/interfaces</code> — network interface configuration for ifup and ifdown
* <code>iw</code> — show/manipulate wireless devices and their configuration
* <code>wpa_cli</code> — WPA command line client
* <code>wpa_supplicant</code> and <code>wpa_supplicant.conf</code> — Wi-Fi Protected Access client and IEEE 802.1X supplicant and its configuration file
* <code>fzf</code> — command line fuzzy finder

=== Use case ===

The use case here is as follow:
* don't connect to a Wi-Fi hotspot unless I explicitly ask the COMPUTAR to do so
* if I need to connect to a Wi-Fi hotspot, the COMPUTAR should give me a list of the visible ones
* a user-friendly™ interface with fuzzy search should help me tell the COMPUTAR which visible hotspot I want to connect to
* once selected and if the Wi-Fi hotspot is already known, the COMPUTAR shall connect to it
* however if the Wi-Fi hotspot is unknown, the COMPUTAR shall kindly ask me for the password, and if I agree to fulfill this request, the COMPUTAR shall save it, and proceed to connect to the hotspot
* Oh no. I am now connected with more COMPUTARS

'''Note:''' if the Wi-Fi hotspot needs to be configured with specific settings (for instance for specific key management settings of for open networks), it is necessary to edit the <code>wpa_supplicant.conf</code> file manually. My rationale is that I need to do this so rarely, that quickly editing a file with a lot of obscure settings outweighs by far the need to make a more complicated shell script to handle such corner case. YMMV :)

== Config and script ==
=== <code>/etc/network/interfaces</code> ===

Edit <code>/etc/network/interfaces</code> and adjust to your network name (here wlp2s0).

<pre>
auto wlp2s0
iface wlp2s0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp
</pre>

=== <code>wpa_supplicant.conf</code> ===

If you start from scratch the only thing you need in this file is this:
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev

If you already have some entries listed (in case you have been using a wifi network manager), make sure to disable each network with <code>disabled=1</code>. This is needed to prevent wpa_supplicant to :

<pre>
# Settings for my nice network
network={
ssid="La Compagnie Créole"
psk="AU BAL AU BAL MASQUÉ OHÉ OHÉ"
disabled=1
}
</pre>

=== The shell script ===

<pre>
#!/bin/sh
set -e

W_IF="wlp2s0" # change with your right interface
WPA_CONF="/etc/wpa_supplicant/wpa_supplicant.conf"

# bogus call of sudo to force auth so as to not be bothered by fzf blocking
# the prompt later on.
# assumes that you're not being prompt for passwd at every single call :)
sudo echo -n

# with a little help from my friends
get_essid ()
{
ESSID=$(sudo iw ${W_IF} scan | rg "\tSSID" | \
sed -e 's/^.*SSID: //g' | sort | uniq | fzf )
}

known_essid ()
{
ID=$(wpa_cli list_networks | rg -F "${ESSID}")
}

connect_essid ()
{
echo "${ESSID} found, connecting..."
echo
wpa_cli enable_network ${ID}
# uncomment following if using with wpa-conf
#echo
#sudo dhclient -v ${W_IF}
}

add_essid ()
{
while true
do
echo "password plz"
read ESSID_PASSWD
while true
do
echo "is \"${ESSID_PASSWD}\" correct?"
read ANSWER_PASSWD
if [ "${ANSWER_PASSWD}" = "y" ]
then
sudo sh -c "cat >> /etc/wpa_supplicant/wpa_supplicant.conf" <<-EOF
# ${ESSID}
network={
ssid="${ESSID}"
psk="${ESSID_PASSWD}"
disabled=1
}

EOF
break
fi
done
break
done
echo "${ESSID} added, connecting..."
}

# Here we go
#cat ~/bin/fox.txt # aesthetics
get_essid
if known_essid
then
connect_essid
break
else
while true
do
echo "dunno, add?"
read ANSWER_ADD
if [ "${ANSWER_ADD}" = "y" ]
then
add_essid
wpa_cli reconfigure
known_essid
connect_essid
break
fi
done
fi
</pre>

== tweaks ==

This setup makes use of the <code>wpa-roam</code> feature of <code>wpa_action</code> under Debian. This means that once connected to an hotspot, the underlying network system will under certain circumstances hop to another access point of the same name/type and, if needed, renegotiate a new IP. Why? Well as the word roam implies, this is handy if you are moving around a building with a bunch of access points all serving the same SSID. If this is not acceptable for whatever reasons, it's possible to prevent this behavior and be connected only to the hotspot/ap you have allowed in the first place. For this you need to edit <code>/etc/network/interfaces</code> like this:

<pre>
auto wlp2s0
iface wlp2s0 inet manual
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp
</pre>

And because you will loose the automatic IP request negotiation, you need to un-comment in the script the two lines that call <code>dhclient</code> in <code>connect_essid ()</code>.

[[Category:Wi-Fi]]

Minimal TUI Wi-Fi Manager

2024-06-16T18:46:02Z

320x200: /* How does it work? */

Good news! If you don't want to install third party daemons, gooey GUI things, or bloat just so you can connect to a wireless access point, then don't worry, Linux has all the CLI tools necessary to create a minimal on-demand TUI shell script thing to connect to, and manage these very Wi-Fi connections. The following info is for Debian, however the approach will be the same for other distros, including BSD operating systems that will differ mostly for some of the network commands. Why would you do this? I dunno, aesthetics, religious beliefs, limited resources, or just an opportunity to better understand what's running in the background.

== How does it work? ==
=== The things that mater ===

* <code>/etc/network/interfaces</code> — network interface configuration for ifup and ifdown
* <code>iw</code> — show/manipulate wireless devices and their configuration
* <code>wpa_cli</code> — WPA command line client
* <code>wpa_supplicant</code> and <code>wpa_supplicant.conf</code> — Wi-Fi Protected Access client and IEEE 802.1X supplicant and its configuration file
* <code>fzf</code> — command line fuzzy finder

=== Use case ===

The use case here is as follow:
* don't connect to a Wi-Fi hotspot unless I explicitly ask the COMPUTAR to do so
* if I need to connect to a Wi-Fi hotspot, the COMPUTAR should give me a list of the visible ones
* a user-friendly™ interface with fuzzy search should help me tell the COMPUTAR which visible hotspot I want to connect to
* once selected and if the Wi-Fi hotspot is already known, the COMPUTAR shall connect to it
* however if the Wi-Fi hotspot is unknown, the COMPUTAR shall kindly ask me for the password, and if I agree to fulfill this request, the COMPUTAR shall save it, and proceed to connect to the hotspot
* Oh no. I am now connected with more COMPUTARS

'''Note:''' if the Wi-Fi hotspot needs to be configured with specific settings (for instance for specific key management settings of for open networks), it is necessary to edit the <code>wpa_supplicant.conf</code> file manually. My rationale is that I need to do this so rarely, that quickly editing a file with a lot of obscure settings outweighs by far the need to make a more complicated shell script to handle such corner case. YMMV :)

== Config and script ==
=== <code>/etc/network/interfaces</code> ===

Edit <code>/etc/network/interfaces</code> and adjust to your network name (here wlp2s0).

<pre>
auto wlp2s0
iface wlp2s0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp
</pre>

=== <code>wpa_supplicant.conf</code> ===

If you start from scratch the only thing you need in this file is this:
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev

If you already have some entries listed (in case you have been using a wifi network manager), make sure to disable each network with <code>disabled=1</code>. This is needed to prevent wpa_supplicant to :

<pre>
# Settings for my nice network
network={
ssid="La Compagnie Créole"
psk="AU BAL AU BAL MASQUÉ OHÉ OHÉ"
disabled=1
}
</pre>

=== The shell script ===

<pre>
#!/bin/sh
set -e

W_IF="wlp2s0" # change with your right interface
WPA_CONF="/etc/wpa_supplicant/wpa_supplicant.conf"

# bogus call of sudo to force auth so as to not be bothered by fzf blocking
# the prompt later on.
# assumes that you're not being prompt for passwd at every single call :)
sudo echo -n

# with a little help from my friends
get_essid ()
{
ESSID=$(sudo iw ${W_IF} scan | rg "\tSSID" | \
sed -e 's/^.*SSID: //g' | sort | uniq | fzf )
}

known_essid ()
{
ID=$(wpa_cli list_networks | rg -F "${ESSID}")
}

connect_essid ()
{
echo "${ESSID} found, connecting..."
echo
wpa_cli enable_network ${ID}
# uncomment following if using with wpa-conf
#echo
#sudo dhclient -v ${W_IF}
}

add_essid ()
{
while true
do
echo "password plz"
read ESSID_PASSWD
while true
do
echo "is \"${ESSID_PASSWD}\" correct?"
read ANSWER_PASSWD
if [ "${ANSWER_PASSWD}" = "y" ]
then
sudo sh -c "cat >> /etc/wpa_supplicant/wpa_supplicant.conf" <<-EOF
# ${ESSID}
network={
ssid="${ESSID}"
psk="${ESSID_PASSWD}"
disabled=1
}

EOF
break
fi
done
break
done
echo "${ESSID} added, connecting..."
}

# Here we go
#cat ~/bin/fox.txt # aesthetics
get_essid
if known_essid
then
connect_essid
break
else
while true
do
echo "dunno, add?"
read ANSWER_ADD
if [ "${ANSWER_ADD}" = "y" ]
then
add_essid
wpa_cli reconfigure
known_essid
connect_essid
break
fi
done
fi
</pre>

== tweaks ==

This setup makes use of the <code>wpa-roam</code> feature of <code>wpa_action</code> under Debian. This means that once connected to an hotspot, the underlying network system will hop to another access point of the same name/type and if needed renegotiate a new IP. This is handy if you are moving around a building with a bunch of access points all serving the same SSID. If this is not acceptable for whatever reasons, it's possible to prevent this behavior and be connected only to the hotspot/ap you have allowed in the first place. For this you need to edit <code>/etc/network/interfaces</code> like this:

<pre>
auto wlp2s0
iface wlp2s0 inet manual
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp
</pre>

And because you will loose the automatic IP request negotiation, you need to un-comment in the script the two lines that call <code>dhclient</code> in <code>connect_essid ()</code>.

[[Category:Wi-Fi]]

Category:Wi-Fi

2024-06-16T18:42:02Z

320x200: Created blank page

Minimal TUI Wi-Fi Manager

2024-06-16T18:41:33Z

320x200:

Good news! If you don't want to install third party daemons, gooey GUI things, or bloat just so you can connect to a wireless access point, then don't worry, Linux has all the CLI tools necessary to create a minimal on-demand TUI shell script thing to connect to, and manage these very Wi-Fi connections. The following info is for Debian, however the approach will be the same for other distros, including BSD operating systems that will differ mostly for some of the network commands. Why would you do this? I dunno, aesthetics, religious beliefs, limited resources, or just an opportunity to better understand what's running in the background.

== How does it work? ==
=== The things that mater ===

* <code>/etc/network/interfaces</code> — network interface configuration for ifup and ifdown
* <code>iw</code> — show/manipulate wireless devices and their configuration
* <code>wpa_cli</code> — WPA command line client
* <code>wpa_supplicant</code> and <code>wpa_supplicant.conf</code> — Wi-Fi Protected Access client and IEEE 802.1X supplicant and its configuration file
* <code>fzf</code> — command line fuzzy finder

=== Use case ===

The use case here is as follow:
* don't connect to a Wi-Fi hotspot unless I explicitly ask the COMPUTAR to do so
* if I need to connect to a Wi-Fi hotspot, the COMPUTAR should give me a list of the visible ones
* a user-friendly™ interface with fuzzy search should help me tell the COMPUTAR which hotspot I want to connect to
* if the Wi-Fi hotspot is already known, the COMPUTAR shall connect to it
* if the Wi-Fi hotspot is unknown, the COMPUTAR shall ask for the password, save it, and connect to the hotspot

'''Note:''' if the Wi-Fi hotspot needs to be configured with specific settings (for instance for specific key management settings of for open networks), it is necessary to edit the <code>wpa_supplicant.conf</code> file manually. My rationale is that I need to do this so rarely, that quickly editing a file with a lot of obscure settings outweighs by far the need to make a more complicated shell script to handle such corner case. YMMV :)

== Config and script ==
=== <code>/etc/network/interfaces</code> ===

Edit <code>/etc/network/interfaces</code> and adjust to your network name (here wlp2s0).

<pre>
auto wlp2s0
iface wlp2s0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp
</pre>

=== <code>wpa_supplicant.conf</code> ===

If you start from scratch the only thing you need in this file is this:
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev

If you already have some entries listed (in case you have been using a wifi network manager), make sure to disable each network with <code>disabled=1</code>. This is needed to prevent wpa_supplicant to :

<pre>
# Settings for my nice network
network={
ssid="La Compagnie Créole"
psk="AU BAL AU BAL MASQUÉ OHÉ OHÉ"
disabled=1
}
</pre>

=== The shell script ===

<pre>
#!/bin/sh
set -e

W_IF="wlp2s0" # change with your right interface
WPA_CONF="/etc/wpa_supplicant/wpa_supplicant.conf"

# bogus call of sudo to force auth so as to not be bothered by fzf blocking
# the prompt later on.
# assumes that you're not being prompt for passwd at every single call :)
sudo echo -n

# with a little help from my friends
get_essid ()
{
ESSID=$(sudo iw ${W_IF} scan | rg "\tSSID" | \
sed -e 's/^.*SSID: //g' | sort | uniq | fzf )
}

known_essid ()
{
ID=$(wpa_cli list_networks | rg -F "${ESSID}")
}

connect_essid ()
{
echo "${ESSID} found, connecting..."
echo
wpa_cli enable_network ${ID}
# uncomment following if using with wpa-conf
#echo
#sudo dhclient -v ${W_IF}
}

add_essid ()
{
while true
do
echo "password plz"
read ESSID_PASSWD
while true
do
echo "is \"${ESSID_PASSWD}\" correct?"
read ANSWER_PASSWD
if [ "${ANSWER_PASSWD}" = "y" ]
then
sudo sh -c "cat >> /etc/wpa_supplicant/wpa_supplicant.conf" <<-EOF
# ${ESSID}
network={
ssid="${ESSID}"
psk="${ESSID_PASSWD}"
disabled=1
}

EOF
break
fi
done
break
done
echo "${ESSID} added, connecting..."
}

# Here we go
#cat ~/bin/fox.txt # aesthetics
get_essid
if known_essid
then
connect_essid
break
else
while true
do
echo "dunno, add?"
read ANSWER_ADD
if [ "${ANSWER_ADD}" = "y" ]
then
add_essid
wpa_cli reconfigure
known_essid
connect_essid
break
fi
done
fi
</pre>

== tweaks ==

This setup makes use of the <code>wpa-roam</code> feature of <code>wpa_action</code> under Debian. This means that once connected to an hotspot, the underlying network system will hop to another access point of the same name/type and if needed renegotiate a new IP. This is handy if you are moving around a building with a bunch of access points all serving the same SSID. If this is not acceptable for whatever reasons, it's possible to prevent this behavior and be connected only to the hotspot/ap you have allowed in the first place. For this you need to edit <code>/etc/network/interfaces</code> like this:

<pre>
auto wlp2s0
iface wlp2s0 inet manual
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp
</pre>

And because you will loose the automatic IP request negotiation, you need to un-comment in the script the two lines that call <code>dhclient</code> in <code>connect_essid ()</code>.

[[Category:Wi-Fi]]

Minimal TUI Wi-Fi Manager

2024-06-16T18:41:07Z

320x200:

Good news! If you don't want to install third party daemons, gooey GUI things, or bloat just so you can connect to a wireless access point, then don't worry, Linux has all the CLI tools necessary to create a minimal on-demand TUI shell script thing to connect to, and manage these very Wi-Fi connections. The following info is for Debian, however the approach will be the same for other distros, including BSD operating systems that will differ mostly for some of the network commands. Why would you do this? I dunno, aesthetics, religious beliefs, limited resources, or just an opportunity to better understand what's running in the background.

== How does it work? ==
=== The things that mater ===

* <code>/etc/network/interfaces</code> — network interface configuration for ifup and ifdown
* <code>iw</code> — show/manipulate wireless devices and their configuration
* <code>wpa_cli</code> — WPA command line client
* <code>wpa_supplicant</code> and <code>wpa_supplicant.conf</code> — Wi-Fi Protected Access client and IEEE 802.1X supplicant and its configuration file
* <code>fzf</code> — command line fuzzy finder

=== Use case ===

The use case here is as follow:
* don't connect to a Wi-Fi hotspot unless I explicitly ask the COMPUTAR to do so
* if I need to connect to a Wi-Fi hotspot, the COMPUTAR should give me a list of the visible ones
* a user-friendly™ interface with fuzzy search should help me tell the COMPUTAR which hotspot I want to connect to
* if the Wi-Fi hotspot is already known, the COMPUTAR shall connect to it
* if the Wi-Fi hotspot is unknown, the COMPUTAR shall ask for the password, save it, and connect to the hotspot

'''Note:''' if the Wi-Fi hotspot needs to be configured with specific settings (for instance for specific key management settings of for open networks), it is necessary to edit the <code>wpa_supplicant.conf</code> file manually. My rationale is that I need to do this so rarely, that quickly editing a file with a lot of obscure settings outweighs by far the need to make a more complicated shell script to handle such corner case. YMMV :)

== Config and script ==
=== <code>/etc/network/interfaces</code> ===

Edit <code>/etc/network/interfaces</code> and adjust to your network name (here wlp2s0).

<pre>
auto wlp2s0
iface wlp2s0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp
</pre>

=== <code>wpa_supplicant.conf</code> ===

If you start from scratch the only thing you need in this file is this:
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev

If you already have some entries listed (in case you have been using a wifi network manager), make sure to disable each network with <code>disabled=1</code>. This is needed to prevent wpa_supplicant to :

<pre>
# Settings for my nice network
network={
ssid="La Compagnie Créole"
psk="AU BAL AU BAL MASQUÉ OHÉ OHÉ"
disabled=1
}
</pre>

=== The shell script ===

<pre>
#!/bin/sh
set -e

W_IF="wlp2s0" # change with your right interface
WPA_CONF="/etc/wpa_supplicant/wpa_supplicant.conf"

# bogus call of sudo to force auth so as to not be bothered by fzf blocking
# the prompt later on.
# assumes that you're not being prompt for passwd at every single call :)
sudo echo -n

# with a little help from my friends
get_essid ()
{
ESSID=$(sudo iw ${W_IF} scan | rg "\tSSID" | \
sed -e 's/^.*SSID: //g' | sort | uniq | fzf )
}

known_essid ()
{
ID=$(wpa_cli list_networks | rg -F "${ESSID}")
}

connect_essid ()
{
echo "${ESSID} found, connecting..."
echo
wpa_cli enable_network ${ID}
# uncomment following if using with wpa-conf
#echo
#sudo dhclient -v ${W_IF}
}

add_essid ()
{
while true
do
echo "password plz"
read ESSID_PASSWD
while true
do
echo "is \"${ESSID_PASSWD}\" correct?"
read ANSWER_PASSWD
if [ "${ANSWER_PASSWD}" = "y" ]
then
sudo sh -c "cat >> /etc/wpa_supplicant/wpa_supplicant.conf" <<-EOF
# ${ESSID}
network={
ssid="${ESSID}"
psk="${ESSID_PASSWD}"
disabled=1
}

EOF
break
fi
done
break
done
echo "${ESSID} added, connecting..."
}

# Here we go
#cat ~/bin/fox.txt # aesthetics
get_essid
if known_essid
then
connect_essid
break
else
while true
do
echo "dunno, add?"
read ANSWER_ADD
if [ "${ANSWER_ADD}" = "y" ]
then
add_essid
wpa_cli reconfigure
known_essid
connect_essid
break
fi
done
fi
</pre>

== tweaks ==

This setup makes use of the <code>wpa-roam</code> feature of <code>wpa_action</code> under Debian. This means that once connected to an hotspot, the underlying network system will hop to another access point of the same name/type and if needed renegotiate a new IP. This is handy if you are moving around a building with a bunch of access points all serving the same SSID. If this is not acceptable for whatever reasons, it's possible to prevent this behavior and be connected only to the hotspot/ap you have allowed in the first place. For this you need to edit <code>/etc/network/interfaces</code> like this:

<pre>
auto wlp2s0
iface wlp2s0 inet manual
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp
</pre>

And because you will loose the automatic IP request negotiation, you need to un-comment in the script the two lines that call <code>dhclient</code> in <code>connect_essid ()</code>.

Minimal TUI Wi-Fi Manager

2024-06-16T18:20:54Z

320x200:

Good news! If you don't want to install third party daemons, gooey GUI things, or bloat just so you can connect to a wireless access point, then don't worry, Linux has all the CLI tools necessary to create a minimal on-demand TUI shell script thing to connect to, and manage these very Wi-Fi connections. The following info is for Debian, however the approach will be the same for other distros, including BSD operating systems that will differ mostly for some of the network commands. Why would you do this? I dunno, aesthetics, religious beliefs, limited resources, or just an opportunity to better understand what's running in the background.

== How does it work? ==
=== The things that mater ===

* <code>/etc/network/interfaces</code> — network interface configuration for ifup and ifdown
* <code>iw</code> — show/manipulate wireless devices and their configuration
* <code>wpa_cli</code> — WPA command line client
* <code>wpa_supplicant</code> and <code>wpa_supplicant.conf</code> — Wi-Fi Protected Access client and IEEE 802.1X supplicant and its configuration file
* <code>fzf</code> — command line fuzzy finder

=== Use case ===

The use case here is as follow:
* don't connect to a Wi-Fi hotspot unless I explicitly ask the COMPUTAR to do so
* if I need to connect to a Wi-Fi hotspot, the COMPUTAR should give me a list of the visible ones
* a user-friendly™ interface with fuzzy search should help me tell the COMPUTAR which hotspot I want to connect to
* if the Wi-Fi hotspot is already known, the COMPUTAR shall connect to it
* if the Wi-Fi hotspot is unknown, the COMPUTAR shall ask for the password, save it, and connect to the hotspot

'''Note:''' if the Wi-Fi hotspot needs to be configured with specific settings (for instance for specific key management settings of for open networks), it is necessary to edit the <code>wpa_supplicant.conf</code> file manually. My rationale is that I need to do this so rarely, that quickly editing a file with a lot of obscure settings outweighs by far the need to make a more complicated shell script to handle such corner case. YMMV :)

== Config and script ==
=== <code>/etc/network/interfaces</code> ===

Edit <code>/etc/network/interfaces</code> and adjust to your network name (here wlp2s0).

<pre>
auto wlp2s0
iface wlp2s0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp
</pre>

=== <code>wpa_supplicant.conf</code> ===

If you start from scratch the only thing you need in this file is this:
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev

If you already have some entries listed (in case you have been using a wifi network manager), make sure to disable each network with <code>disabled=1</code>. This is needed to prevent wpa_supplicant to :

<pre>
# Settings for my nice network
network={
ssid="La Compagnie Créole"
psk="AU BAL AU BAL MASQUÉ OHÉ OHÉ"
disabled=1
}
</pre>

=== The shell script ===

<pre>
#!/bin/sh
set -e

W_IF="wlp2s0" # change with your right interface
WPA_CONF="/etc/wpa_supplicant/wpa_supplicant.conf"

# bogus call of sudo to force auth so as to not be bothered by fzf blocking
# the prompt later on.
# assumes that you're not being prompt for passwd at every single call :)
sudo echo -n

# with a little help from my friends
get_essid ()
{
ESSID=$(sudo iw ${W_IF} scan | rg "\tSSID" | \
sed -e 's/^.*SSID: //g' | sort | uniq | fzf )
}

known_essid ()
{
ID=$(wpa_cli list_networks | rg -F "${ESSID}")
}

connect_essid ()
{
echo "${ESSID} found, connecting..."
echo
wpa_cli enable_network ${ID}
# uncomment following if using with wpa-run
#echo
#sudo dhclient -v ${W_IF}
}

add_essid ()
{
while true
do
echo "password plz"
read ESSID_PASSWD
while true
do
echo "is \"${ESSID_PASSWD}\" correct?"
read ANSWER_PASSWD
if [ "${ANSWER_PASSWD}" = "y" ]
then
sudo sh -c "cat >> /etc/wpa_supplicant/wpa_supplicant.conf" <<-EOF
# ${ESSID}
network={
ssid="${ESSID}"
psk="${ESSID_PASSWD}"
disabled=1
}

EOF
break
fi
done
break
done
echo "${ESSID} added, connecting..."
}

# Here we go
#cat ~/bin/fox.txt # aesthetics
get_essid
if known_essid
then
connect_essid
break
else
while true
do
echo "dunno, add?"
read ANSWER_ADD
if [ "${ANSWER_ADD}" = "y" ]
then
add_essid
wpa_cli reconfigure
known_essid
connect_essid
break
fi
done
fi
</pre>

== tweaks ==

Minimal TUI Wi-Fi Manager

2024-06-16T17:11:42Z

320x200:

Minimal TUI Wi-Fi Manager

2024-06-16T16:51:30Z

320x200: /* Use case */

Minimal TUI Wi-Fi Manager

2024-06-16T16:48:46Z

320x200: /* Config and script */

Minimal TUI Wi-Fi Manager

2024-06-16T16:43:49Z

320x200: /* How does it work? */

Minimal TUI Wi-Fi Manager

2024-06-16T16:35:09Z

320x200:

Minimal TUI Wi-Fi Manager

2024-06-16T16:24:45Z

320x200: /* The things that mater */

Minimal TUI Wi-Fi Manager

2024-06-16T16:22:55Z

320x200: Created page with "Good news! If you don't want to install third party daemons, gooey GUI things, or bloat just so you can connect to a wireless access point, then don't worry, Linux has all the..."

Main Page

2024-05-02T22:16:36Z

320x200:

[[File:Waymo.webp|right|512px]]
'''Welcome to RUN YOUR OWN, the wiki documentation of LURK's servers.'''

One of LURK's mission is to make sure we're not creating yet another silo for online discussions and publishing, but also give enough inspiration for others to do the same and start provide their own centralised, decentralised, federated, and whatnot services.

Running your own server, and its services, is not always straightforward, so this is why we are using this space to document and make public how we maintain and configure LURK servers (as well as some of the machines we use).

'''This documentation effort is an ongoing process, we add stuff as we develop the environment, and learn more about it ourselves.'''

== Server Documents ==

{{Special:AllPages|namespace=14}}
* [[TODO]]

Firewall

2024-05-02T22:14:50Z

320x200: /* iptables oneliners */

Different ways to handle <code>iptables</code>.

== Using <code>iptables-persistent</code> on Debian ==

'''Note:''' In use on <code>vrijdagmiddagborrel</code>.

It's basically a set of <code>iptables</code> plugins for <code>netfilter-persistent</code>, which itself is a loader for different netfilter configuration. Once installed, it will take care of restoring rules at boot time, and through a small helper, can be used to reload/update/save rules on the fly.

=== Installation and config ===
* Installation:
apt install iptables-persistent netfilter-persistent
* Add/change iptables rules located at <code>/etc/iptables/rules.v4</code> and <code>/etc/iptables/rules.v6</code>

=== Usage ===
* Apply new rules after changes made to <code>rules.v*</code> files and check result
netfilter-persistent reload
iptables -L

== <code>iptables</code> oneliners ==
* list all rules from all chains
iptables -L
* block an IP
iptables -I INPUT -s 192.168.111.111 -j DROP
iptables -I OUTPUT -d 192.168.111.111 -j DROP

[[Category:System]]

Main Page

2024-05-02T22:13:34Z

320x200:

[[File:Waymo.webp|right|512px]]
'''Welcome to RUN YOUR OWN, the wiki documentation of LURK's servers.'''

One of LURK's mission is to make sure we're not creating yet another silo for online discussions and publishing, but also give enough inspiration for others to do the same and start provide their own centralised, decentralised, federated, and whatnot services.

Running your own server, and its services, is not always straightforward, so this is why we are using this space to document and make public how we maintain and configure LURK servers (as well as some of the machines we use).

== Request for Feedback ==

This documentation effort is an ongoing process, we add stuff as we develop the environment, and learn more about it ourselves. We gladly welcome your feedback and comments:
* Mailing list about LURK: https://we.lurk.org/postorius/lists/metalurk.we.lurk.org
* Chat about LURK: https://talk.lurk.org/channel/_meta

== Server Documents ==

{{Special:AllPages|namespace=14}}
* [[TODO]]

Main Page

2024-05-02T22:11:57Z

320x200:

[[File:Waymo.webp|right]]
'''Welcome to RUN YOUR OWN, the wiki documentation of LURK's servers.'''

One of LURK's mission is to make sure we're not creating yet another silo for online discussions and publishing, but also give enough inspiration for others to do the same and start provide their own centralised, decentralised, federated, and whatnot services.

Running your own server, and its services, is not always straightforward, so this is why we are using this space to document and make public how we maintain and configure LURK servers (as well as some of the machines we use).

== Request for Feedback ==

This documentation effort is an ongoing process, we add stuff as we develop the environment, and learn more about it ourselves. We gladly welcome your feedback and comments:
* Mailing list about LURK: https://we.lurk.org/postorius/lists/metalurk.we.lurk.org
* Chat about LURK: https://talk.lurk.org/channel/_meta

== Server Documents ==

{{Special:AllPages|namespace=14}}
* [[TODO]]

File:Waymo.webp

2024-05-02T22:11:21Z

320x200:

Category:Maintenance

2024-04-05T13:59:13Z

320x200: Created blank page

Uninstall systemd services

2024-04-05T13:59:04Z

320x200:

<pre>
systemctl stop [servicename]
systemctl disable [servicename]
rm /etc/systemd/system/[servicename]
rm /etc/systemd/system/[servicename] # and symlinks that might be related
rm /usr/lib/systemd/system/[servicename]
rm /usr/lib/systemd/system/[servicename] # and symlinks that might be related
systemctl daemon-reload
systemctl reset-failed
</pre>

[[Category: Maintenance]]

Uninstall systemd services

2024-04-05T13:58:27Z

320x200: Created page with "<pre> systemctl stop [servicename] systemctl disable [servicename] rm /etc/systemd/system/[servicename] rm /etc/systemd/system/[servicename] # and symlinks that might be relat..."

Cleanup

2024-04-04T05:32:08Z

320x200: Created page with "Sometimes you need to cleanup. Because there is too much shit on your disks. === general cruft === <code>ncdu</code> is your friend. <code>dust</code> too. === log files ===..."

Sometimes you need to cleanup. Because there is too much shit on your disks.

=== general cruft ===
<code>ncdu</code> is your friend. <code>dust</code> too.

=== log files ===
==== /var/log ====
Easy. just <code>rm</code> compressed archives and the first <code>.1</code> rotated files. If you are really in emergency mode you can also delete the live log files, they will be recreated, don't worry.

==== systemd managed logs ====
For these it's better to use the dedicated tool:
journalctl --disk-usage # what's the damage
journalctl --vacuum-size=200M # delete old logs so that current logs only take 10MB disk usage

[[Category:Storage]]

Wildcard Certificates with acme.sh

2024-04-02T08:27:23Z

320x200:

== Using acme.sh ==

Since the certbot gandi dns plugin has been giving issues over the past months (no smooth renewal, leading to unavailability of sites) we started moving things to acme.sh. Seems to work better, easier to setup etc.

=== Install the bash script ===

wget https://get.acme.sh

As root:

sh acme.sh

This will install the script to /root/ and add it to path

=== Request a wildcard cert for lurk.org ===

first find and export the gandi dns key:

export GANDI_LIVEDNS_KEY="fdmlfsdklmfdkmqsdfk"

Then request a wildcard cert. (the dns key is added to a config file automatically for future renewals)

acme.sh --issue --dns dns_gandi_livedns --nginx -d *.lurk.org

Find the certs in:

/root/.acme.sh/\*.lurk.org/

== Migration status ==

Done:

* VMB (nginx, masto, icecast)

To do:

* agnes (nginx, prosody, mm3)
* douglas (???)

== Ref ==

* https://forums.freebsd.org/threads/howto-ssl-tls-certificates-with-acme-sh.61231/ (FreeBSD)

= N.B. everything below is for archival reference =

== Base installation ==

apt install python-pip
pip install wheel
pip install certbot

== DNS plugins ==
=== Gandi ===
'''NOTE: At time of writing, only an API key from the domain owner will work. So another account, even if listed technical contact, will not able to use the live DNS API, just the live DNS web interface. Since the writing of this HOWTO, there is now this as well https://github.com/obynio/certbot-plugin-gandi TODO'''

* Get API key from Gandi (somewhere in account settings)
* install certbot-plugin-gandi
pip install 'git+https://gitlab.com/cspublic/certbot-plugin-gandi.git'
mkdir /etc/certbot-plugin-gandi
* create /etc/certbot-plugin-gandi/gandi.ini with the following:
certbot_plugin_gandi:dns_api_key=APIKEY
* a bit of paranoia
chmod 600 /etc/certbot-plugin-gandi/gandi.ini
* request certificate for both mydomain.blabla '''and''' *.mydomain.blabla
'''NOTE: At time of writing, the default sever end point used by cerbot (0.22) is not compatible with ACME v2, as a workaround --server must be passed manually. Next version of certbot should point to the right server'''
/usr/local/bin/certbot certonly -a certbot-plugin-gandi:dns --certbot-plugin-gandi:dns-credentials /etc/certbot-plugin-gandi/gandi.ini -d mydomain.blabla -d *.mydomain.blabla --server https://acme-v02.api.letsencrypt.org/directory
* If all goes well certs will be there:
'''NOTE: At time of writing, certbot-plugin-gandi seems to behave a bit funnily when asked to request a challenge for a wildcard cert (it works flawslessly for regular domains). It might be needed to run the command several times to get the infamous CONGRATULATION message from certbot.'''
/etc/letsencrypt/live/mydomain.blabla/fullchain.pem
/etc/letsencrypt/live/mydomain.blabla/privkey.pem

== Renewal ==
To non-interactively renew *all* of your certificates:
/usr/local/bin/certbot renew

=== douglas ===
=== agnes ===
'''TODO: hooks!'''
<pre>
service nginx restart
prosodyctl reload
</pre>

[[Category:Certificates]]

Simple LAN filesharing with WebDAV

2024-03-18T23:16:18Z

320x200: /* Pro Tips */

'''WebDAV''' is both a quite popular and yet overlooked way to access and edit files remotely across a wide range of operating systems. Yes it's web stuff, again, but surprisingly fast, lightweight, and that can recover quite well on unstable networks or when the server has to be restarted, or has gone for lunch. A reason why it may be overlooked is possibly because it's often associated with sausage factories like own/nextcloud, or standalone implementations that are not particularly exciting. What is less known is that many web servers come with their own '''WebDAV''' implementation. Out of the usual suspects, '''nginx''', '''Apache''', and '''lighttpd''', the latter has both the most lightweight and most complete implementation. No need for anything else!

In these notes we only cover a simple LAN setup, you can build upon it for more complex use case of course.

== Server side ==
=== Installation ===
* This is for Debian, but I know you're smart, you will figure it out for <code>${FLAVOUR_OF_THE_MONTH_DISTRO}</code>
sudo apt install lighttpd lighttpd-mod-webdav

=== Example Configuration ===
Basically the configuration that matters for now are in <code>/etc/lighttpd/conf-available</code> and with symlinks in <code>/etc/lighttpd/conf-enabled</code>. '''In our simple example we will have three shared folders, one that can be mounted read-only by anyone, and two that are read-write but require a username and password'''.

* By default a temp config file called <code>99-unconfigured.conf</code> provides a generic landing page. We don't need it and we just have to enable the authentication config.
sudo lighttpd-disable-mod unconfigured
sudo lighttpd-enable-mod auth
* Create a user and password for the read-write share
sudo apt install apache2-utils
sudo htpasswd -c /etc/lighttpd/user.htpasswd turtleprincess heygirl
* create a new configuration file <code>/etc/lighttpd/conf-available/66-webdav.conf</code> with the following:
<pre>
server.modules += ( "mod_webdav" )
dir-listing.encoding = "utf-8"

# This is needed for keepings tracks of locks and props which
# are needed for shares that can be edited
webdav.sqlite-db-name = "/var/cache/lighttpd/lighttpd.webdav.db"

# auth
server.modules += ("mod_authn_file")
auth.backend = "htpasswd"
auth.backend.htpasswd.userfile = "/etc/lighttpd/user.htpasswd"
auth.require = ( "/readwrite1" => ( "method" => "basic",
"realm" => "YOU WOT MATE",
"require" => "valid-user" ),
"/readwrite2" => ( "method" => "basic",
"realm" => "YOU WOT MATE",
"require" => "valid-user" ))

# read-only stuff
$HTTP["url"] =~ "^/readonly(?:/|$)" {
alias.url = ( "/readonly" => "/local/path/to/readonly" )
dir-listing.activate = "enable"
webdav.activate = "enable"
webdav.is-readonly = "enable"
}

# shared pit of madness
# for users listed in /etc/lighttpd/user.htpasswd
$HTTP["url"] =~ "^/readwrite1(?:/|$)" {
alias.url = ( "/readwrite1" => "/local/path/to/readwrite1" )
dir-listing.activate = "enable"
webdav.activate = "enable"
webdav.is-readonly = "disable"
}
$HTTP["url"] =~ "^/readwrite2(?:/|$)" {
alias.url = ( "/readwrite2" => "/local/path/to/readwrite2" )
dir-listing.activate = "enable"
webdav.activate = "enable"
webdav.is-readonly = "disable"
}
</pre>

Now if you wonder what kind of magic will happen so that the lighttpd process, local and remote users can happily live together ever after, and edit the same files, well, none. It won't happen just like this, it does not work and you will be sad. Then you will start binge drinking to forget about server administration. You will go in the streets, pick up fights with strangers who are in a much better physical condition because they don't spend hours configuring neovim plugins that ''you haven't even used once''. It will be painful. So to avoid this unfortunate situation you need to do two things:

* First, the folder needs to be group-owned by the lighttpd process owner, in our case, on Debian, it's <code>www-data</code>. You also need to set the group ID bit on the folder you intend to use as WedDAV share, so that all newly created files will inherit the group ownership of this folder.
sudo chown regular_user:www-data /local/path/to/readwrite1
sudo chmod g+ws /local/path/to/readwrite1
* Second, you need to change the way lighttpd is started so that its umask is set in a way that any permission may be set for user and group (default is just user). On a systemd based system (haters gonna hate) you need to edit <code>/etc/systemd/system/lighttpd.service</code> like this:
ExecStart=/bin/sh -c 'umask 002;/usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf'

That's it, you have a working simple LAN filesharing with WebDAV check the documentation for more options, and more fun config time, at home, where it's much safer for you.

== Client side ==
=== Web browser ===
The nice advantage of the setup above is that all the files can be readily browsed at the local IP of the machine running lighttpd, for instance http://192.168.100.100/readonly. If you're into web stuff have a look at https://github.com/dom111/webdav-js, it would be really trivial to serve this js file with the lighttpd server and provide more functionality than just browsing and downloading.

=== Linux filesystem ===
Mounting a WebDAV on Linux is straightforward with <code>davfs2</code> and requires minimal configuration. <code>davfs2</code> is also able to recover quite well from disconnections and potential read/write fuckery (there is a daemon and a local cache, but you don't need to worry about this). If the server is dead, that you need to poweroff your machine, and you still have something mounted, it's a good idea to unmount things first to speed up shutdown. No it won't lock like a little NFS shit but there is a timeout and you're a busy person.
* To mount manually the read-only folder from above:
sudo apt install davfs2
sudo mount -t davfs http://192.168.100.100/readonly /mnt/readonly
* To remember username passwords you can create a personal <code>~/davfs2/secrets</code> file with:
http://192.168.100.100/readwrite1 turtleprincess heygirl
http://192.168.100.100/readwrite2 turtleprincess heygirl
etc
* You can also have everything configured in your <code>/etc/fstab</code> and then mount everything as a regular user:
<pre>
# My cool LAN WebDAV thing
http://192.168.100.100/readonly /mnt/readonly/ davfs user,uid=username,noauto 0 0
http://192.168.100.100/readwrite1 /mnt/readonly/ davfs user,uid=username,noauto 0 0
http://192.168.100.100/readwrite2 /mnt/readonly/ davfs user,uid=username,noauto 0 0
</pre>

=== FreeBSD filesystem ===
On FreeBSD it's basically the same as on Linux. lol. Of course not! What did you think? That would be too easy. You need to work hard to earn your condescending smug BSD coolness. At time of writing, <code>davfs2</code> is being [https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=267518 ported to FreeBSD by its author]. Don't hold your breath, but hopefully this will happen because the FreeBSD usual way to mount WebDAV stuff with <code>mount.webdavfs</code> just sucks ("Device not configured" on certain reading operations, not able to recover when the server disconnects). As a workaround, the all-you-can-eat-software-buffet <code>rclone</code> is the only reliable way to have WebDAV mounted in a FreeBSD filesystem.

* installation
pkg install rclone # yeah yeah you can use portmaster and disable stuff you will regret later
* load fuse
sudo kldload fusefs
* to make it permanent add the following to <code>/boot/loader.conf</code>
# Filesystems in Userspace
fusefs_load="YES
* create a <code>~/.config/rclone/rclone.conf</code> config file with:
<pre>
[192.168.100.100]
type = webdav
url = http://192.168.100.100/
vendor = other
user = turtleprincess
pass = SEE_BELOW
</pre>
* generate the password with this:
echo "heygirl" | rclone obscure -
* test if it works
sudo mkdir /mnt/readwrite2
sudo chown user:user /mnt/readwrite2
rclone mount 192.168.100.100:readwrite2 /mnt/readwrite2/ --vfs-cache-mode writes --daemon

You can also make entries in your <code>/etc/fstab</code>
* rclone can't really be used directly, so we will trick fuse
sudo ln -s /usr/local/bin/rclone /usr/local/bin/mount.rclone
* in <code>/etc/fstab</code>
<pre>
# My cool LAN WebDAV thing
192.168.100.100:readonly /mnt/readonly fuse noauto,ro,mountprog=/usr/local/bin/mount.rclone
192.168.100.100:readwrite1 /mnt/readwrite1 fuse noauto,rw,mountprog=/usr/local/bin/mount.rclone
192.168.100.100:readwrite1 /mnt/readwrite1 fuse noauto,rw,mountprog=/usr/local/bin/mount.rclone
</pre>

=== Nautilus ===
For those into GUI (not judging, it's a free country) WebDAV is really easy to access thanks to the GVfs daemon. It's also a solid option in terms of recovery and making sure little to no fuckery is possible when writing on an unstable network. In practice, click on the "other" location button on the left pane, choose WedDAV and enter the URL in the form of <code>dav://192.168.100.100/readwrite1</code> and give your credentials. The only caveat is that the mounted folder name will be the server name, only the server name (IP in this case), so you may want to add your share to the bookmarks, and rename the bookmark to something more meaningful.

=== Android ===
Several commercial, closed source options. We don't talk to these people. There are however three interesting FLOSS options:
* https://github.com/newhinton/Round-Sync - super complete, multiple network file storage supported (based on rclone)
* https://github.com/phpbg/easysync - minimal automated WebDAV sync for the usual Android user data folders
* https://github.com/bitfireAT/davx5-ose - focused on CalDAV/CardDAV but hidden in one menu there is a mini browsing option for WebDAV

=== Nintendo Switch ===
Because you know, reasons:
* https://github.com/cy33hc/switch-ezremote-client - simple file browser/editor
* https://github.com/J-D-K/JKSV - popular save file manager, a WebDAV folder can be the default folder to export save backups

== Pro Tips ==
* IPs are used above but you can use a local DNS or make use of your local <code>/etc/hosts</code> files to give a nice name to your WedDAV server, look into RFC 8375 for pointers.
* To debug things if things are not behaving as nice things:
lighttpd-enable-mod accesslog
systemctl restart lighttpd
tail -f /var/log/lighttpd/access.log
# 17.4 hours later
# Problem solved at last
lighttpd-disable-mod accesslog
systemctl restart lighttpd

[[Category: WebDAV]]

Category:WebDAV

2024-03-18T23:15:49Z

320x200: Created blank page

Android WebDAV Bi-directional sync with Termux and Rclone

2024-03-18T23:15:36Z

320x200:

The purpose of this fantastic documentation is to explain how to setup a very lightweight on-demand manual bi-directional sync between your Android device and a WebDAV share hosted somewhere. It's fast, and you trade appsolutionism for a cool shell script that you can execute from the comfort of your very own home screen. This is all possible because of Termux, a terminal emulator application for Android that comes with its apt based package manager, and Rclone, a command-line program to manage files on way too many different cloud storage systems. Seriously it's ridiculous.

=== Warnings ===

* This documentation makes some assumptions regarding WebDAV and its configuration [[Simple_LAN_filesharing_with_WebDAV | that are described there]].
* Bi-Directional sync in Rclone is considered experimental at time of writing. See https://rclone.org/bisync/ for more info.
* This is not a Termux or Rclone tutorial, RTFM or GTFO :)

== The Setup ==
=== Prerequisites ===
* Install Termux with F-Droid
* Make your life easier and access Termux via ssh, read this https://wiki.termux.com/wiki/Remote_Access#Using_the_SSH_server
* From Termux
pkg upgrade
pkg install rclone
* feel free to also install quality of life extras like <code>tmux</code>, <code>neovim</code>, etc
* configure rclone to work with your WebDAV share, create <code>~/.config/rclone/rclone.conf</code>
[library]
type = webdav
url = http://192.168.100.100/
vendor = other
user = turtleprincess
pass = SEE_BELOW
* generate the password with this:
echo "heygirl" | rclone obscure -
* test if it works with
rclone lsd library:some/folder

=== Preparing the local and remote folders ===
Here we will take as example that you want to maintain in sync two folders, the default <code>Pictures</code> folder on your Android device and a remote WebDAV folder of the same name, that can be anywhere in your remote share, for instance <code>library:path/to/Pictures</code>. '''Make sure to make a backup of your Android <code>Pictures</code> folder in case <code>${SOMETHING_B4D}</code> happens. If you fuck it up, it's on you. Also follow these three steps below otherwise you will make things worse anyway.'''

* First we need to test what happens if we ask to populate everything on both sides (content on both local and remote folder will form a superset, so everything will be on both sides)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync --dry-run
* If this looks it's working fine, then it's time to stop pretending (drop <code>--dry-run</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync
* After that, no need to maintain dumb superset sync (drop <code>--resync</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case

If all went well, then you can use the last oneliner whenever you change something on your Android device or on the remote share, and the changes will be propagated to the other. But starting Termux everytime to type this command can be a PITA, except when you're in public transport and you want to cosplay like you're breaking into a gibson. So let's use a script.

== The script ==
=== shhhhhhh ===
The following script can sync several folders at the root of the Android user storage, each of them will be treated separately, just make sure that:
* they also exist on the remote WebDAV share
* for each of these folders you have done the little 3-steps dance described above

The script has a Wake-on-LAN section, feel free to get rid of it, I sync only on LAN because WAN is overrated and it's cold outside anyway.

<pre>
#!/data/data/com.termux/files/usr/bin/sh

set -e # exit on error

DAVIP="192.168.100.100"
DAVMAC="c0:de:d3:4d:c0:de"
DAVFOLDER="library:path/to"
ANDFOLDERS="DCIM Pictures Download Documents"
RETRY=0

kthxbye ()
{
echo "Press a key to exit..."
read REPLY
exit 0
}

syncit ()
{
echo "library is open!\n"
# ANSI art greetings (optional but you know you want it)
#/data/data/com.termux/files/usr/bin/printf "$(cat ~/assets/tp.ans)"
#echo
#sleep 3s
for ANDFOLDER in ${ANDFOLDERS}
do
echo "syncing ***${ANDFOLDER}***\n"
rclone bisync /sdcard/${ANDFOLDER} ${DAVFOLDER}/${ANDFOLDER} \
--create-empty-src-dirs --compare size --slow-hash-sync-only \
--resilient -MvP --fix-case
done
kthxbye
}

gotolibrary ()
{
if nc -w 2 -z ${DAVIP} 80 2>/dev/null
then
syncit
else
echo "library is silent, trying to wakeup turtle princess"
if [ "${RETRY}" -eq 0 ] ; then wol ${DAVMAC} ; fi
while [ "${RETRY}" -lt 20 ]
do
sleep 1s
RETRY=$((RETRY+1))
gotolibrary
done
echo "OK I give up, library is closed :("
kthxbye
fi
}

gotolibrary
</pre>

Save it with the very original name <code>sync.sh</code> and you're good to go.

'''BUT WAIT THERE'S MORE'''

=== Widget ===
The script is cool but you know what's more cool? To be able to start it directly from the home launcher thing.
* Install Termux-widget from F-Droid, an add-on app which adds shortcuts to commands on the home screen.
* From Termux, move the script to a new special folder
mkdir .shortcuts
mv sync.sh .shortcuts
* Go to your home screen and long press to add a Widget, scroll through hundreds of crap widgets, and eventually select the Termux widget.
* once dropped, <code>sync.sh</code> should automagically be present in the widget menu, or you can also use the single icon widget if you prefer that
* now you can call the script directly from the home screen.
* The future is now.

== Notes ==
* files with ":" in their name won't sync because they can't be created on an Android fs
* we can't use modtime as a criteria for detecting file changes so filesize is the main criteria. We can't use it because Android does not support this somehow. I'm a man of simple taste, I just need to sync 10GB of memes. Bitrot and tempering can only add value to such endeavor. Check Rclone bisync doc for more options if you find this proposal outrageous.
* access to <code>/sdcard</code> can stop working sometimes, because Android. Just revoke Termux storage permission and enable them again
* when the script is launched from the home screen it's nice to be able to use the whole height of the Android device to scroll back through all these error messages, but by default Termux puts a software keyboard in your face, you can politely decline by adding the following in <code>.termux/termux.properties</code>
hide-soft-keyboard-on-startup = true

[[Category: WebDAV]]

Android WebDAV Bi-directional sync with Termux and Rclone

2024-03-18T23:12:00Z

320x200: /* Preparing the local and remote folders */

Android WebDAV Bi-directional sync with Termux and Rclone

2024-03-18T23:05:49Z

320x200:

The purpose of this fantastic documentation is to explain how to setup a very lightweight on-demand manual bi-directional sync between your Android device and a WebDAV share hosted somewhere. It's fast, and you trade appsolutionism for a cool shell script that you can execute from the comfort of your very own home screen. This is all possible because of Termux, a terminal emulator application for Android that comes with its apt based package manager, and Rclone, a command-line program to manage files on way too many different cloud storage systems. Seriously it's ridiculous.

=== Warnings ===

* This documentation makes some assumptions regarding WebDAV and its configuration [[Simple_LAN_filesharing_with_WebDAV | that are described there]].
* Bi-Directional sync in Rclone is considered experimental at time of writing. See https://rclone.org/bisync/ for more info.
* This is not a Termux or Rclone tutorial, RTFM or GTFO :)

== The Setup ==
=== Prerequisites ===
* Install Termux with F-Droid
* Make your life easier and access Termux via ssh, read this https://wiki.termux.com/wiki/Remote_Access#Using_the_SSH_server
* From Termux
pkg upgrade
pkg install rclone
* feel free to also install quality of life extras like <code>tmux</code>, <code>neovim</code>, etc
* configure rclone to work with your WebDAV share, create <code>~/.config/rclone/rclone.conf</code>
[library]
type = webdav
url = http://192.168.100.100/
vendor = other
user = turtleprincess
pass = SEE_BELOW
* generate the password with this:
echo "heygirl" | rclone obscure -
* test if it works with
rclone lsd library:some/folder

=== Preparing the local and remote folders ===
Here we will take as example that you want to maintain in sync two folders, the default <code>Pictures</code> folder on your Android device and a remote WebDAV folder of the same name, that can be anywhere in your remote share, for instance <code>library:path/to/Pictures</code>. '''Make sure to make a backup of your Android <code>Pictures</code> folder in case <code>${SOMETHING_B4D}</code> happens. If you fuck it up, it's on you. Also follow these three steps below otherwise you will make things worse anyway.'''

* First we need to test what happens if we ask to populate everything on both sides (content on both local and remote folder will form a superset, so everything will be on both sides)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync --dry-run
* If this looks it's working fine, then times to stop pretending (drop <code>--dry-run</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync
* After that, no need to maintain dumb superset sync (drop <code>--resync</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case

If all went well, then you can use the last oneliner whenever you change something on your Android device or on the remote share, and the changes will be propagated to the other. But starting Termux everytime to type this command can be a PITA, except when you're in public transport and you want to cosplay like you're breaking into a gibson. So let's use a script.

== The script ==
=== shhhhhhh ===
The following script can sync several folders at the root of the Android user storage, each of them will be treated separately, just make sure that:
* they also exist on the remote WebDAV share
* for each of these folders you have done the little 3-steps dance described above

The script has a Wake-on-LAN section, feel free to get rid of it, I sync only on LAN because WAN is overrated and it's cold outside anyway.

<pre>
#!/data/data/com.termux/files/usr/bin/sh

set -e # exit on error

DAVIP="192.168.100.100"
DAVMAC="c0:de:d3:4d:c0:de"
DAVFOLDER="library:path/to"
ANDFOLDERS="DCIM Pictures Download Documents"
RETRY=0

kthxbye ()
{
echo "Press a key to exit..."
read REPLY
exit 0
}

syncit ()
{
echo "library is open!\n"
# ANSI art greetings (optional but you know you want it)
#/data/data/com.termux/files/usr/bin/printf "$(cat ~/assets/tp.ans)"
#echo
#sleep 3s
for ANDFOLDER in ${ANDFOLDERS}
do
echo "syncing ***${ANDFOLDER}***\n"
rclone bisync /sdcard/${ANDFOLDER} ${DAVFOLDER}/${ANDFOLDER} \
--create-empty-src-dirs --compare size --slow-hash-sync-only \
--resilient -MvP --fix-case
done
kthxbye
}

gotolibrary ()
{
if nc -w 2 -z ${DAVIP} 80 2>/dev/null
then
syncit
else
echo "library is silent, trying to wakeup turtle princess"
if [ "${RETRY}" -eq 0 ] ; then wol ${DAVMAC} ; fi
while [ "${RETRY}" -lt 20 ]
do
sleep 1s
RETRY=$((RETRY+1))
gotolibrary
done
echo "OK I give up, library is closed :("
kthxbye
fi
}

gotolibrary
</pre>

Save it with the very original name <code>sync.sh</code> and you're good to go.

'''BUT WAIT THERE'S MORE'''

=== Widget ===
The script is cool but you know what's more cool? To be able to start it directly from the home launcher thing.
* Install Termux-widget from F-Droid, an add-on app which adds shortcuts to commands on the home screen.
* From Termux, move the script to a new special folder
mkdir .shortcuts
mv sync.sh .shortcuts
* Go to your home screen and long press to add a Widget, scroll through hundreds of crap widgets, and eventually select the Termux widget.
* once dropped, <code>sync.sh</code> should automagically be present in the widget menu, or you can also use the single icon widget if you prefer that
* now you can call the script directly from the home screen.
* The future is now.

== Notes ==
* files with ":" in their name won't sync because they can't be created on an Android fs
* we can't use modtime as a criteria for detecting file changes so filesize is the main criteria. We can't use it because Android does not support this somehow. I'm a man of simple taste, I just need to sync 10GB of memes. Bitrot and tempering can only add value to such endeavor. Check Rclone bisync doc for more options if you find this proposal outrageous.
* access to <code>/sdcard</code> can stop working sometimes, because Android. Just revoke Termux storage permission and enable them again
* when the script is launched from the home screen it's nice to be able to use the whole height of the Android device to scroll back through all these error messages, but by default Termux puts a software keyboard in your face, you can politely decline by adding the following in <code>.termux/termux.properties</code>
hide-soft-keyboard-on-startup = true

Android WebDAV Bi-directional sync with Termux and Rclone

2024-03-18T23:05:17Z

320x200: /* Notes */

The purpose of this fantastic documentation is to explain how to setup a very lightweight on-demand manual bi-directional sync between your Android device and a WebDAV share hosted somewhere. It's fast, and you trade appsolutionism for a cool shell script that you can execute from the comfort of your very own home launcher. This is all possible because of Termux, a terminal emulator application for Android that comes with its apt based package manager, and Rclone, a command-line program to manage files on way too many different cloud storage systems. Seriously it's ridiculous.

=== Warnings ===

* This documentation makes some assumptions regarding WebDAV and its configuration [[Simple_LAN_filesharing_with_WebDAV | that are described there]].
* Bi-Directional sync in Rclone is considered experimental at time of writing. See https://rclone.org/bisync/ for more info.
* This is not a Termux or Rclone tutorial, RTFM or GTFO :)

== The Setup ==
=== Prerequisites ===
* Install Termux with F-Droid
* Make your life easier and access Termux via ssh, read this https://wiki.termux.com/wiki/Remote_Access#Using_the_SSH_server
* From Termux
pkg upgrade
pkg install rclone
* feel free to also install quality of life extras like <code>tmux</code>, <code>neovim</code>, etc
* configure rclone to work with your WebDAV share, create <code>~/.config/rclone/rclone.conf</code>
[library]
type = webdav
url = http://192.168.100.100/
vendor = other
user = turtleprincess
pass = SEE_BELOW
* generate the password with this:
echo "heygirl" | rclone obscure -
* test if it works with
rclone lsd library:some/folder

=== Preparing the local and remote folders ===
Here we will take as example that you want to maintain in sync two folders, the default <code>Pictures</code> folder on your Android device and a remote WebDAV folder of the same name, that can be anywhere in your remote share, for instance <code>library:path/to/Pictures</code>. '''Make sure to make a backup of your Android <code>Pictures</code> folder in case <code>${SOMETHING_B4D}</code> happens. If you fuck it up, it's on you. Also follow these three steps below otherwise you will make things worse anyway.'''

* First we need to test what happens if we ask to populate everything on both sides (content on both local and remote folder will form a superset, so everything will be on both sides)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync --dry-run
* If this looks it's working fine, then times to stop pretending (drop <code>--dry-run</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync
* After that, no need to maintain dumb superset sync (drop <code>--resync</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case

If all went well, then you can use the last oneliner whenever you change something on your Android device or on the remote share, and the changes will be propagated to the other. But starting Termux everytime to type this command can be a PITA, except when you're in public transport and you want to cosplay like you're breaking into a gibson. So let's use a script.

== The script ==
=== shhhhhhh ===
The following script can sync several folders at the root of the Android user storage, each of them will be treated separately, just make sure that:
* they also exist on the remote WebDAV share
* for each of these folders you have done the little 3-steps dance described above

The script has a Wake-on-LAN section, feel free to get rid of it, I sync only on LAN because WAN is overrated and it's cold outside anyway.

<pre>
#!/data/data/com.termux/files/usr/bin/sh

set -e # exit on error

DAVIP="192.168.100.100"
DAVMAC="c0:de:d3:4d:c0:de"
DAVFOLDER="library:path/to"
ANDFOLDERS="DCIM Pictures Download Documents"
RETRY=0

kthxbye ()
{
echo "Press a key to exit..."
read REPLY
exit 0
}

syncit ()
{
echo "library is open!\n"
# ANSI art greetings (optional but you know you want it)
#/data/data/com.termux/files/usr/bin/printf "$(cat ~/assets/tp.ans)"
#echo
#sleep 3s
for ANDFOLDER in ${ANDFOLDERS}
do
echo "syncing ***${ANDFOLDER}***\n"
rclone bisync /sdcard/${ANDFOLDER} ${DAVFOLDER}/${ANDFOLDER} \
--create-empty-src-dirs --compare size --slow-hash-sync-only \
--resilient -MvP --fix-case
done
kthxbye
}

gotolibrary ()
{
if nc -w 2 -z ${DAVIP} 80 2>/dev/null
then
syncit
else
echo "library is silent, trying to wakeup turtle princess"
if [ "${RETRY}" -eq 0 ] ; then wol ${DAVMAC} ; fi
while [ "${RETRY}" -lt 20 ]
do
sleep 1s
RETRY=$((RETRY+1))
gotolibrary
done
echo "OK I give up, library is closed :("
kthxbye
fi
}

gotolibrary
</pre>

Save it with the very original name <code>sync.sh</code> and you're good to go.

'''BUT WAIT THERE'S MORE'''

=== Widget ===
The script is cool but you know what's more cool? To be able to start it directly from the home launcher thing.
* Install Termux-widget from F-Droid, an add-on app which adds shortcuts to commands on the home screen.
* From Termux, move the script to a new special folder
mkdir .shortcuts
mv sync.sh .shortcuts
* Go to your home screen and long press to add a Widget, scroll through hundreds of crap widgets, and eventually select the Termux widget.
* once dropped, <code>sync.sh</code> should automagically be present in the widget menu, or you can also use the single icon widget if you prefer that
* now you can call the script directly from the home screen.
* The future is now.

== Notes ==
* files with ":" in their name won't sync because they can't be created on an Android fs
* we can't use modtime as a criteria for detecting file changes so filesize is the main criteria. We can't use it because Android does not support this somehow. I'm a man of simple taste, I just need to sync 10GB of memes. Bitrot and tempering can only add value to such endeavor. Check Rclone bisync doc for more options if you find this proposal outrageous.
* access to <code>/sdcard</code> can stop working sometimes, because Android. Just revoke Termux storage permission and enable them again
* when the script is launched from the home screen it's nice to be able to use the whole height of the Android device to scroll back through all these error messages, but by default Termux puts a software keyboard in your face, you can politely decline by adding the following in <code>.termux/termux.properties</code>
hide-soft-keyboard-on-startup = true

Android WebDAV Bi-directional sync with Termux and Rclone

2024-03-18T23:04:58Z

320x200: /* Notes */

The purpose of this fantastic documentation is to explain how to setup a very lightweight on-demand manual bi-directional sync between your Android device and a WebDAV share hosted somewhere. It's fast, and you trade appsolutionism for a cool shell script that you can execute from the comfort of your very own home launcher. This is all possible because of Termux, a terminal emulator application for Android that comes with its apt based package manager, and Rclone, a command-line program to manage files on way too many different cloud storage systems. Seriously it's ridiculous.

=== Warnings ===

* This documentation makes some assumptions regarding WebDAV and its configuration [[Simple_LAN_filesharing_with_WebDAV | that are described there]].
* Bi-Directional sync in Rclone is considered experimental at time of writing. See https://rclone.org/bisync/ for more info.
* This is not a Termux or Rclone tutorial, RTFM or GTFO :)

== The Setup ==
=== Prerequisites ===
* Install Termux with F-Droid
* Make your life easier and access Termux via ssh, read this https://wiki.termux.com/wiki/Remote_Access#Using_the_SSH_server
* From Termux
pkg upgrade
pkg install rclone
* feel free to also install quality of life extras like <code>tmux</code>, <code>neovim</code>, etc
* configure rclone to work with your WebDAV share, create <code>~/.config/rclone/rclone.conf</code>
[library]
type = webdav
url = http://192.168.100.100/
vendor = other
user = turtleprincess
pass = SEE_BELOW
* generate the password with this:
echo "heygirl" | rclone obscure -
* test if it works with
rclone lsd library:some/folder

=== Preparing the local and remote folders ===
Here we will take as example that you want to maintain in sync two folders, the default <code>Pictures</code> folder on your Android device and a remote WebDAV folder of the same name, that can be anywhere in your remote share, for instance <code>library:path/to/Pictures</code>. '''Make sure to make a backup of your Android <code>Pictures</code> folder in case <code>${SOMETHING_B4D}</code> happens. If you fuck it up, it's on you. Also follow these three steps below otherwise you will make things worse anyway.'''

* First we need to test what happens if we ask to populate everything on both sides (content on both local and remote folder will form a superset, so everything will be on both sides)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync --dry-run
* If this looks it's working fine, then times to stop pretending (drop <code>--dry-run</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync
* After that, no need to maintain dumb superset sync (drop <code>--resync</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case

If all went well, then you can use the last oneliner whenever you change something on your Android device or on the remote share, and the changes will be propagated to the other. But starting Termux everytime to type this command can be a PITA, except when you're in public transport and you want to cosplay like you're breaking into a gibson. So let's use a script.

== The script ==
=== shhhhhhh ===
The following script can sync several folders at the root of the Android user storage, each of them will be treated separately, just make sure that:
* they also exist on the remote WebDAV share
* for each of these folders you have done the little 3-steps dance described above

The script has a Wake-on-LAN section, feel free to get rid of it, I sync only on LAN because WAN is overrated and it's cold outside anyway.

<pre>
#!/data/data/com.termux/files/usr/bin/sh

set -e # exit on error

DAVIP="192.168.100.100"
DAVMAC="c0:de:d3:4d:c0:de"
DAVFOLDER="library:path/to"
ANDFOLDERS="DCIM Pictures Download Documents"
RETRY=0

kthxbye ()
{
echo "Press a key to exit..."
read REPLY
exit 0
}

syncit ()
{
echo "library is open!\n"
# ANSI art greetings (optional but you know you want it)
#/data/data/com.termux/files/usr/bin/printf "$(cat ~/assets/tp.ans)"
#echo
#sleep 3s
for ANDFOLDER in ${ANDFOLDERS}
do
echo "syncing ***${ANDFOLDER}***\n"
rclone bisync /sdcard/${ANDFOLDER} ${DAVFOLDER}/${ANDFOLDER} \
--create-empty-src-dirs --compare size --slow-hash-sync-only \
--resilient -MvP --fix-case
done
kthxbye
}

gotolibrary ()
{
if nc -w 2 -z ${DAVIP} 80 2>/dev/null
then
syncit
else
echo "library is silent, trying to wakeup turtle princess"
if [ "${RETRY}" -eq 0 ] ; then wol ${DAVMAC} ; fi
while [ "${RETRY}" -lt 20 ]
do
sleep 1s
RETRY=$((RETRY+1))
gotolibrary
done
echo "OK I give up, library is closed :("
kthxbye
fi
}

gotolibrary
</pre>

Save it with the very original name <code>sync.sh</code> and you're good to go.

'''BUT WAIT THERE'S MORE'''

=== Widget ===
The script is cool but you know what's more cool? To be able to start it directly from the home launcher thing.
* Install Termux-widget from F-Droid, an add-on app which adds shortcuts to commands on the home screen.
* From Termux, move the script to a new special folder
mkdir .shortcuts
mv sync.sh .shortcuts
* Go to your home screen and long press to add a Widget, scroll through hundreds of crap widgets, and eventually select the Termux widget.
* once dropped, <code>sync.sh</code> should automagically be present in the widget menu, or you can also use the single icon widget if you prefer that
* now you can call the script directly from the home screen.
* The future is now.

== Notes ==
* files with ":" in their name won't sync because they can't be created on an Android fs
* we can't use modtime as a criteria for detecting file changes so filesize is the main criteria. We can't use it because Android does not support this somehow. I'm a man of simple taste, I just need to sync 10GB of memes. Bitrot and tempering can only add value to such endeavor. Check Rclone bisync doc for more options if you find this proposal outrageous.
* access to <code>/sdcard<code> can stop working sometimes, because Android. Just revoke Termux storage permission and enable them again
* when the script is launched from the home screen it's nice to be able to use the whole height of the Android device to scroll back through all these error messages, but by default Termux puts a software keyboard in your face, you can politely decline by adding the following in <code>.termux/termux.properties</code>
hide-soft-keyboard-on-startup = true

Android WebDAV Bi-directional sync with Termux and Rclone

2024-03-18T22:36:49Z

320x200: /* Widget */

The purpose of this fantastic documentation is to explain how to setup a very lightweight on-demand manual bi-directional sync between your Android device and a WebDAV share hosted somewhere. It's fast, and you trade appsolutionism for a cool shell script that you can execute from the comfort of your very own home launcher. This is all possible because of Termux, a terminal emulator application for Android that comes with its apt based package manager, and Rclone, a command-line program to manage files on way too many different cloud storage systems. Seriously it's ridiculous.

=== Warnings ===

* This documentation makes some assumptions regarding WebDAV and its configuration [[Simple_LAN_filesharing_with_WebDAV | that are described there]].
* Bi-Directional sync in Rclone is considered experimental at time of writing. See https://rclone.org/bisync/ for more info.
* This is not a Termux or Rclone tutorial, RTFM or GTFO :)

== The Setup ==
=== Prerequisites ===
* Install Termux with F-Droid
* Make your life easier and access Termux via ssh, read this https://wiki.termux.com/wiki/Remote_Access#Using_the_SSH_server
* From Termux
pkg upgrade
pkg install rclone
* feel free to also install quality of life extras like <code>tmux</code>, <code>neovim</code>, etc
* configure rclone to work with your WebDAV share, create <code>~/.config/rclone/rclone.conf</code>
[library]
type = webdav
url = http://192.168.100.100/
vendor = other
user = turtleprincess
pass = SEE_BELOW
* generate the password with this:
echo "heygirl" | rclone obscure -
* test if it works with
rclone lsd library:some/folder

=== Preparing the local and remote folders ===
Here we will take as example that you want to maintain in sync two folders, the default <code>Pictures</code> folder on your Android device and a remote WebDAV folder of the same name, that can be anywhere in your remote share, for instance <code>library:path/to/Pictures</code>. '''Make sure to make a backup of your Android <code>Pictures</code> folder in case <code>${SOMETHING_B4D}</code> happens. If you fuck it up, it's on you. Also follow these three steps below otherwise you will make things worse anyway.'''

* First we need to test what happens if we ask to populate everything on both sides (content on both local and remote folder will form a superset, so everything will be on both sides)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync --dry-run
* If this looks it's working fine, then times to stop pretending (drop <code>--dry-run</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync
* After that, no need to maintain dumb superset sync (drop <code>--resync</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case

If all went well, then you can use the last oneliner whenever you change something on your Android device or on the remote share, and the changes will be propagated to the other. But starting Termux everytime to type this command can be a PITA, except when you're in public transport and you want to cosplay like you're breaking into a gibson. So let's use a script.

== The script ==
=== shhhhhhh ===
The following script can sync several folders at the root of the Android user storage, each of them will be treated separately, just make sure that:
* they also exist on the remote WebDAV share
* for each of these folders you have done the little 3-steps dance described above

The script has a Wake-on-LAN section, feel free to get rid of it, I sync only on LAN because WAN is overrated and it's cold outside anyway.

<pre>
#!/data/data/com.termux/files/usr/bin/sh

set -e # exit on error

DAVIP="192.168.100.100"
DAVMAC="c0:de:d3:4d:c0:de"
DAVFOLDER="library:path/to"
ANDFOLDERS="DCIM Pictures Download Documents"
RETRY=0

kthxbye ()
{
echo "Press a key to exit..."
read REPLY
exit 0
}

syncit ()
{
echo "library is open!\n"
# ANSI art greetings (optional but you know you want it)
#/data/data/com.termux/files/usr/bin/printf "$(cat ~/assets/tp.ans)"
#echo
#sleep 3s
for ANDFOLDER in ${ANDFOLDERS}
do
echo "syncing ***${ANDFOLDER}***\n"
rclone bisync /sdcard/${ANDFOLDER} ${DAVFOLDER}/${ANDFOLDER} \
--create-empty-src-dirs --compare size --slow-hash-sync-only \
--resilient -MvP --fix-case
done
kthxbye
}

gotolibrary ()
{
if nc -w 2 -z ${DAVIP} 80 2>/dev/null
then
syncit
else
echo "library is silent, trying to wakeup turtle princess"
if [ "${RETRY}" -eq 0 ] ; then wol ${DAVMAC} ; fi
while [ "${RETRY}" -lt 20 ]
do
sleep 1s
RETRY=$((RETRY+1))
gotolibrary
done
echo "OK I give up, library is closed :("
kthxbye
fi
}

gotolibrary
</pre>

Save it with the very original name <code>sync.sh</code> and you're good to go.

'''BUT WAIT THERE'S MORE'''

=== Widget ===
The script is cool but you know what's more cool? To be able to start it directly from the home launcher thing.
* Install Termux-widget from F-Droid, an add-on app which adds shortcuts to commands on the home screen.
* From Termux, move the script to a new special folder
mkdir .shortcuts
mv sync.sh .shortcuts
* Go to your home screen and long press to add a Widget, scroll through hundreds of crap widgets, and eventually select the Termux widget.
* once dropped, <code>sync.sh</code> should automagically be present in the widget menu, or you can also use the single icon widget if you prefer that
* now you can call the script directly from the home screen.
* The future is now.

== Notes ==
* files with :
* modtime maybe filesize is the best way
* access to sdcard
* remove keyboard

Android WebDAV Bi-directional sync with Termux and Rclone

2024-03-18T22:36:34Z

320x200: /* The script */

The purpose of this fantastic documentation is to explain how to setup a very lightweight on-demand manual bi-directional sync between your Android device and a WebDAV share hosted somewhere. It's fast, and you trade appsolutionism for a cool shell script that you can execute from the comfort of your very own home launcher. This is all possible because of Termux, a terminal emulator application for Android that comes with its apt based package manager, and Rclone, a command-line program to manage files on way too many different cloud storage systems. Seriously it's ridiculous.

=== Warnings ===

* This documentation makes some assumptions regarding WebDAV and its configuration [[Simple_LAN_filesharing_with_WebDAV | that are described there]].
* Bi-Directional sync in Rclone is considered experimental at time of writing. See https://rclone.org/bisync/ for more info.
* This is not a Termux or Rclone tutorial, RTFM or GTFO :)

== The Setup ==
=== Prerequisites ===
* Install Termux with F-Droid
* Make your life easier and access Termux via ssh, read this https://wiki.termux.com/wiki/Remote_Access#Using_the_SSH_server
* From Termux
pkg upgrade
pkg install rclone
* feel free to also install quality of life extras like <code>tmux</code>, <code>neovim</code>, etc
* configure rclone to work with your WebDAV share, create <code>~/.config/rclone/rclone.conf</code>
[library]
type = webdav
url = http://192.168.100.100/
vendor = other
user = turtleprincess
pass = SEE_BELOW
* generate the password with this:
echo "heygirl" | rclone obscure -
* test if it works with
rclone lsd library:some/folder

=== Preparing the local and remote folders ===
Here we will take as example that you want to maintain in sync two folders, the default <code>Pictures</code> folder on your Android device and a remote WebDAV folder of the same name, that can be anywhere in your remote share, for instance <code>library:path/to/Pictures</code>. '''Make sure to make a backup of your Android <code>Pictures</code> folder in case <code>${SOMETHING_B4D}</code> happens. If you fuck it up, it's on you. Also follow these three steps below otherwise you will make things worse anyway.'''

* First we need to test what happens if we ask to populate everything on both sides (content on both local and remote folder will form a superset, so everything will be on both sides)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync --dry-run
* If this looks it's working fine, then times to stop pretending (drop <code>--dry-run</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync
* After that, no need to maintain dumb superset sync (drop <code>--resync</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case

If all went well, then you can use the last oneliner whenever you change something on your Android device or on the remote share, and the changes will be propagated to the other. But starting Termux everytime to type this command can be a PITA, except when you're in public transport and you want to cosplay like you're breaking into a gibson. So let's use a script.

== The script ==
=== shhhhhhh ===
The following script can sync several folders at the root of the Android user storage, each of them will be treated separately, just make sure that:
* they also exist on the remote WebDAV share
* for each of these folders you have done the little 3-steps dance described above

The script has a Wake-on-LAN section, feel free to get rid of it, I sync only on LAN because WAN is overrated and it's cold outside anyway.

<pre>
#!/data/data/com.termux/files/usr/bin/sh

set -e # exit on error

DAVIP="192.168.100.100"
DAVMAC="c0:de:d3:4d:c0:de"
DAVFOLDER="library:path/to"
ANDFOLDERS="DCIM Pictures Download Documents"
RETRY=0

kthxbye ()
{
echo "Press a key to exit..."
read REPLY
exit 0
}

syncit ()
{
echo "library is open!\n"
# ANSI art greetings (optional but you know you want it)
#/data/data/com.termux/files/usr/bin/printf "$(cat ~/assets/tp.ans)"
#echo
#sleep 3s
for ANDFOLDER in ${ANDFOLDERS}
do
echo "syncing ***${ANDFOLDER}***\n"
rclone bisync /sdcard/${ANDFOLDER} ${DAVFOLDER}/${ANDFOLDER} \
--create-empty-src-dirs --compare size --slow-hash-sync-only \
--resilient -MvP --fix-case
done
kthxbye
}

gotolibrary ()
{
if nc -w 2 -z ${DAVIP} 80 2>/dev/null
then
syncit
else
echo "library is silent, trying to wakeup turtle princess"
if [ "${RETRY}" -eq 0 ] ; then wol ${DAVMAC} ; fi
while [ "${RETRY}" -lt 20 ]
do
sleep 1s
RETRY=$((RETRY+1))
gotolibrary
done
echo "OK I give up, library is closed :("
kthxbye
fi
}

gotolibrary
</pre>

Save it with the very original name <code>sync.sh</code> and you're good to go.

'''BUT WAIT THERE'S MORE'''

=== Widget ===
The script is cool but you know what's more cool? To be able to start it directly from the home launcher thing.
* Install Termux-widget from F-Droid, an add-on app which adds shortcuts to commands on the home screen.
* From Termux, move the script to a new special folder
mkdir .shortcuts
mv sync.sh .shortcuts
* Go to your home screen and long press to add a Widget, scroll through hundreds of crap widgets, and eventually select the Termux widget.
* once dropped, sync.sh should automagically be present in the widget menu, or you can also use the single icon widget if you prefer that
* now you can call the script directly from the home screen.
* The future is now.

== Notes ==
* files with :
* modtime maybe filesize is the best way
* access to sdcard
* remove keyboard

Android WebDAV Bi-directional sync with Termux and Rclone

2024-03-18T22:25:55Z

320x200: /* shhhhhhh */

The purpose of this fantastic documentation is to explain how to setup a very lightweight on-demand manual bi-directional sync between your Android device and a WebDAV share hosted somewhere. It's fast, and you trade appsolutionism for a cool shell script that you can execute from the comfort of your very own home launcher. This is all possible because of Termux, a terminal emulator application for Android that comes with its apt based package manager, and Rclone, a command-line program to manage files on way too many different cloud storage systems. Seriously it's ridiculous.

=== Warnings ===

* This documentation makes some assumptions regarding WebDAV and its configuration [[Simple_LAN_filesharing_with_WebDAV | that are described there]].
* Bi-Directional sync in Rclone is considered experimental at time of writing. See https://rclone.org/bisync/ for more info.
* This is not a Termux or Rclone tutorial, RTFM or GTFO :)

== The Setup ==
=== Prerequisites ===
* Install Termux with F-Droid
* Make your life easier and access Termux via ssh, read this https://wiki.termux.com/wiki/Remote_Access#Using_the_SSH_server
* From Termux
pkg upgrade
pkg install rclone
* feel free to also install quality of life extras like <code>tmux</code>, <code>neovim</code>, etc
* configure rclone to work with your WebDAV share, create <code>~/.config/rclone/rclone.conf</code>
[library]
type = webdav
url = http://192.168.100.100/
vendor = other
user = turtleprincess
pass = SEE_BELOW
* generate the password with this:
echo "heygirl" | rclone obscure -
* test if it works with
rclone lsd library:some/folder

=== Preparing the local and remote folders ===
Here we will take as example that you want to maintain in sync two folders, the default <code>Pictures</code> folder on your Android device and a remote WebDAV folder of the same name, that can be anywhere in your remote share, for instance <code>library:path/to/Pictures</code>. '''Make sure to make a backup of your Android <code>Pictures</code> folder in case <code>${SOMETHING_B4D}</code> happens. If you fuck it up, it's on you. Also follow these three steps below otherwise you will make things worse anyway.'''

* First we need to test what happens if we ask to populate everything on both sides (content on both local and remote folder will form a superset, so everything will be on both sides)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync --dry-run
* If this looks it's working fine, then times to stop pretending (drop <code>--dry-run</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync
* After that, no need to maintain dumb superset sync (drop <code>--resync</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case

If all went well, then you can use the last oneliner whenever you change something on your Android device or on the remote share, and the changes will be propagated to the other. But starting Termux everytime to type this command can be a PITA, except when you're in public transport and you want to cosplay like you're breaking into a gibson. So let's use a script.

== The script ==
=== shhhhhhh ===
The following script can sync several folders at the root of the Android user storage, each of them will be treated separately, just make sure that:
* they also exist on the remote WebDAV share
* for each of these folders you have done the little 3-steps dance described above

The script has a Wake-on-LAN section, feel free to get rid of it, I sync only on LAN because WAN is overrated and it's cold outside anyway.

<pre>
#!/data/data/com.termux/files/usr/bin/sh

set -e # exit on error

DAVIP="192.168.100.100"
DAVMAC="c0:de:d3:4d:c0:de"
DAVFOLDER="library:path/to"
ANDFOLDERS="DCIM Pictures Download Documents"
RETRY=0

kthxbye ()
{
echo "Press a key to exit..."
read REPLY
exit 0
}

syncit ()
{
echo "library is open!\n"
# ANSI art greetings (optional but you know you want it)
#/data/data/com.termux/files/usr/bin/printf "$(cat ~/assets/tp.ans)"
#echo
#sleep 3s
for ANDFOLDER in ${ANDFOLDERS}
do
echo "syncing ***${ANDFOLDER}***\n"
rclone bisync /sdcard/${ANDFOLDER} ${DAVFOLDER}/${ANDFOLDER} \
--create-empty-src-dirs --compare size --slow-hash-sync-only \
--resilient -MvP --fix-case
done
kthxbye
}

gotolibrary ()
{
if nc -w 2 -z ${DAVIP} 80 2>/dev/null
then
syncit
else
echo "library is silent, trying to wakeup turtle princess"
if [ "${RETRY}" -eq 0 ] ; then wol ${DAVMAC} ; fi
while [ "${RETRY}" -lt 20 ]
do
sleep 1s
RETRY=$((RETRY+1))
gotolibrary
done
echo "OK I give up, library is closed :("
kthxbye
fi
}

gotolibrary
</pre>

== Notes ==
* files with :
* modtime maybe filesize is the best way
* access to sdcard
* remove keyboard

Android WebDAV Bi-directional sync with Termux and Rclone

2024-03-18T22:25:08Z

320x200: /* shhhhhhh */

The purpose of this fantastic documentation is to explain how to setup a very lightweight on-demand manual bi-directional sync between your Android device and a WebDAV share hosted somewhere. It's fast, and you trade appsolutionism for a cool shell script that you can execute from the comfort of your very own home launcher. This is all possible because of Termux, a terminal emulator application for Android that comes with its apt based package manager, and Rclone, a command-line program to manage files on way too many different cloud storage systems. Seriously it's ridiculous.

=== Warnings ===

* This documentation makes some assumptions regarding WebDAV and its configuration [[Simple_LAN_filesharing_with_WebDAV | that are described there]].
* Bi-Directional sync in Rclone is considered experimental at time of writing. See https://rclone.org/bisync/ for more info.
* This is not a Termux or Rclone tutorial, RTFM or GTFO :)

== The Setup ==
=== Prerequisites ===
* Install Termux with F-Droid
* Make your life easier and access Termux via ssh, read this https://wiki.termux.com/wiki/Remote_Access#Using_the_SSH_server
* From Termux
pkg upgrade
pkg install rclone
* feel free to also install quality of life extras like <code>tmux</code>, <code>neovim</code>, etc
* configure rclone to work with your WebDAV share, create <code>~/.config/rclone/rclone.conf</code>
[library]
type = webdav
url = http://192.168.100.100/
vendor = other
user = turtleprincess
pass = SEE_BELOW
* generate the password with this:
echo "heygirl" | rclone obscure -
* test if it works with
rclone lsd library:some/folder

=== Preparing the local and remote folders ===
Here we will take as example that you want to maintain in sync two folders, the default <code>Pictures</code> folder on your Android device and a remote WebDAV folder of the same name, that can be anywhere in your remote share, for instance <code>library:path/to/Pictures</code>. '''Make sure to make a backup of your Android <code>Pictures</code> folder in case <code>${SOMETHING_B4D}</code> happens. If you fuck it up, it's on you. Also follow these three steps below otherwise you will make things worse anyway.'''

* First we need to test what happens if we ask to populate everything on both sides (content on both local and remote folder will form a superset, so everything will be on both sides)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync --dry-run
* If this looks it's working fine, then times to stop pretending (drop <code>--dry-run</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync
* After that, no need to maintain dumb superset sync (drop <code>--resync</code>)
rclone bisync /sdcard/Pictures library:path/to/Pictures --create-empty-src-dirs --compare size --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case

If all went well, then you can use the last oneliner whenever you change something on your Android device or on the remote share, and the changes will be propagated to the other. But starting Termux everytime to type this command can be a PITA, except when you're in public transport and you want to cosplay like you're breaking into a gibson. So let's use a script.

== The script ==
=== shhhhhhh ===
The following script can sync several folders at the root of the Android user storage, each of them will be treated separately, just make sure that:
* they also exist on the remote WebDAV share
* for each of these folders you have done the little 3-steps dance described above

The script has a Wake-on-LAN section, feel free to get rid of it, I sync only on LAN because WAN is overrated and it's cold outside anyway.

<pre>
#!/data/data/com.termux/files/usr/bin/sh

set -e # exit on error

DAVIP="192.168.100.100"
DAVMAC="c0:de:d3:4d:c0:de"
DAVFOLDER="library:path/to"
ANDFOLDERS="DCIM Pictures Download Documents"
RETRY=0

kthxbye ()
{
echo "Press a key to exit..."
read REPLY
exit 0
}

syncit ()
{
echo "library is open!\n"
# ANSI art greetings (optional but you know you want it)
#/data/data/com.termux/files/usr/bin/printf "$(cat ~/assets/tp.ans)"
#echo
#sleep 3s
for ANDFOLDER in ${ANDFOLDERS}
do
echo "syncing ***${ANDFOLDER}***\n"
rclone bisync /sdcard/${ANDFOLDER} ${DAVFOLDER}/${ANDFOLDER} \
--create-empty-src-dirs --compare size --slow-hash-sync-only \
--resilient -MvP --fix-case
done
kthxbye
}

gotolibrary ()
{
if nc -w 2 -z ${DAVIP} 80 2>/dev/null
then
syncit
else
echo "library is silent, trying to wakeup turtle princess"
if [ "${RETRY}" -eq 0 ] ; then wol ${DAVMAC} ; fi
while [ "${RETRY}" -lt 20 ]
do
sleep 1s
RETRY=$((RETRY+1))
gotolibrary
done
echo "OK I give up, library is closed :("
kthxbye
fi
}

gotolibrary
</pre>

== Notes ==
* files with :
* modtime maybe filesize is the best way
* access to sdcard
* remove keyboard