Jekyll2023-07-21T04:21:23-05:00http://blog.angrydev.net/feed.xmlangrydev blogthe life and crisis of a c# developerangrydev blogwebmaster@angrydev.netusing virsh and systemd to automatically size ballooning device of kvm guest2023-07-19T06:51:00-05:002023-07-19T06:51:00-05:00http://blog.angrydev.net/using-virsh-and-systemd-to-automatically-size-ballooning-device-of-kvm-guest<p>on my primary machine that is currently using fedora i have a windows guest for some testing stuff that at some point needs more ram to do some tasks.
to support that scenario i added a ballooning device and installed the qemu / virtio guest tools.
at this point i’m able to manually adjust the memory allocation of the vm using the balloning driver using virsh:</p>
<p><code class="language-plaintext highlighter-rouge">sudo virsh setmem "win" 4G</code></p>
<p>now it would be really useful if this could be done automatically. i searched for existing solutions and found a <a href="https://www.linux-kvm.org/page/Projects/auto-ballooning">kvm project that was abandoned 2013</a> …</p>
<p>so we need a script, here is my first rough approach, installed as a systemd service:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
<span class="c"># can be intsalled as a systemd service via</span>
<span class="c"># sudo systemctl edit --full --force virshautomem</span>
<span class="c">#</span>
<span class="c"># [Unit]</span>
<span class="c"># Description=virsh auto memory allocation for ballooning vms</span>
<span class="c">#</span>
<span class="c"># [Service]</span>
<span class="c"># ExecStart=/usr/local/bin/virshautomem.sh -d</span>
<span class="c">#</span>
<span class="c"># [Install]</span>
<span class="c"># WantedBy=default.target</span>
<span class="c"># minimum memory to assign</span>
<span class="nv">MIN_MEMORY</span><span class="o">=</span><span class="k">$((</span><span class="m">2</span> <span class="o">*</span> <span class="m">1024</span> <span class="o">*</span> <span class="m">1024</span><span class="k">))</span>
<span class="c"># lower and upper threshold in percent of complete assigned memory</span>
<span class="nv">LOWTHRES</span><span class="o">=</span>5
<span class="nv">UPTHRES</span><span class="o">=</span>10
<span class="c"># Retrieve the connection URI</span>
<span class="c"># uri=$(virsh uri)</span>
<span class="nv">uri</span><span class="o">=</span><span class="s2">"qemu:///system"</span>
<span class="nv">daemon</span><span class="o">=</span>0
<span class="k">if</span> <span class="o">[[</span> <span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"-d"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
</span><span class="nv">daemon</span><span class="o">=</span>1<span class="p">;</span>
<span class="nb">echo</span> <span class="s2">"running as daemon"</span>
<span class="k">fi
while</span>
<span class="c"># Retrieve a list of running VMs</span>
<span class="nv">running_vms</span><span class="o">=</span><span class="si">$(</span>virsh <span class="nt">--connect</span> <span class="s2">"</span><span class="nv">$uri</span><span class="s2">"</span> list <span class="nt">--name</span> <span class="nt">--state-running</span><span class="si">)</span>
<span class="c"># echo "checking running vms: $running_vms"</span>
<span class="c"># Iterate over each VM</span>
<span class="k">for </span>vm_name <span class="k">in</span> <span class="nv">$running_vms</span><span class="p">;</span> <span class="k">do</span>
<span class="c"># Retrieve VM XML configuration</span>
<span class="nv">vm_xml</span><span class="o">=</span><span class="si">$(</span>virsh <span class="nt">--connect</span> <span class="s2">"</span><span class="nv">$uri</span><span class="s2">"</span> dumpxml <span class="s2">"</span><span class="nv">$vm_name</span><span class="s2">"</span><span class="si">)</span>
<span class="c"># echo "checking $vm_name"</span>
<span class="c"># echo "$vm_xml"</span>
<span class="c"># Check if the VM has ballooning support</span>
<span class="k">if </span><span class="nb">echo</span> <span class="s2">"</span><span class="nv">$vm_xml</span><span class="s2">"</span> | <span class="nb">grep</span> <span class="nt">-q</span> <span class="s1">'<memballoon'</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="nt">-n</span> <span class="s2">"adjusting memory for VM: </span><span class="nv">$vm_name</span><span class="s2">: "</span>
<span class="c"># Extract maximum memory from VM info using xmllint and XPath</span>
<span class="nv">max_memory_value</span><span class="o">=</span><span class="si">$(</span><span class="nb">echo</span> <span class="s2">"</span><span class="nv">$vm_xml</span><span class="s2">"</span> | xmllint <span class="nt">--xpath</span> <span class="s1">'string(/domain/memory)'</span> -<span class="si">)</span>
<span class="nv">max_memory_unit</span><span class="o">=</span><span class="si">$(</span><span class="nb">echo</span> <span class="s2">"</span><span class="nv">$vm_xml</span><span class="s2">"</span> | xmllint <span class="nt">--xpath</span> <span class="s1">'string(/domain/memory/@unit)'</span> -<span class="si">)</span>
<span class="c"># Convert max_memory_value to kilobytes</span>
<span class="k">case</span> <span class="nv">$max_memory_unit</span> <span class="k">in
</span>k <span class="p">|</span> kb <span class="p">|</span> KiB<span class="p">)</span>
<span class="nv">MAX_MEMORY</span><span class="o">=</span><span class="nv">$max_memory_value</span>
<span class="p">;;</span>
m <span class="p">|</span> mb <span class="p">|</span> MiB<span class="p">)</span>
<span class="nv">MAX_MEMORY</span><span class="o">=</span><span class="k">$((</span><span class="nv">$max_memory_value</span> <span class="o">*</span> <span class="m">1024</span><span class="k">))</span>
<span class="p">;;</span>
g <span class="p">|</span> gb <span class="p">|</span> GiB<span class="p">)</span>
<span class="nv">MAX_MEMORY</span><span class="o">=</span><span class="k">$((</span><span class="nv">$max_memory_value</span> <span class="o">*</span> <span class="m">1024</span> <span class="o">*</span> <span class="m">1024</span><span class="k">))</span>
<span class="p">;;</span>
t <span class="p">|</span> tb<span class="p">)</span>
<span class="nv">MAX_MEMORY</span><span class="o">=</span><span class="k">$((</span><span class="nv">$max_memory_value</span> <span class="o">*</span> <span class="m">1024</span> <span class="o">*</span> <span class="m">1024</span> <span class="o">*</span> <span class="m">1024</span><span class="k">))</span>
<span class="p">;;</span>
<span class="k">*</span><span class="p">)</span>
<span class="nb">echo</span> <span class="s2">"Unsupported memory unit: </span><span class="nv">$max_memory_unit</span><span class="s2">"</span>
<span class="k">continue</span>
<span class="p">;;</span>
<span class="k">esac</span>
<span class="c"># Retrieve memory statistics</span>
<span class="nv">mem_stat</span><span class="o">=</span><span class="si">$(</span>virsh <span class="nt">--connect</span> <span class="s2">"</span><span class="nv">$uri</span><span class="s2">"</span> dommemstat <span class="s2">"</span><span class="nv">$vm_name</span><span class="s2">"</span><span class="si">)</span>
<span class="nv">current_memory</span><span class="o">=</span><span class="si">$(</span><span class="nb">echo</span> <span class="s2">"</span><span class="nv">$mem_stat</span><span class="s2">"</span> | <span class="nb">awk</span> <span class="s1">'/actual/ { print $2 }'</span><span class="si">)</span>
<span class="nv">available_memory</span><span class="o">=</span><span class="si">$(</span><span class="nb">echo</span> <span class="s2">"</span><span class="nv">$mem_stat</span><span class="s2">"</span> | <span class="nb">awk</span> <span class="s1">'/usable/ { print $2 }'</span><span class="si">)</span>
<span class="c"># Calculate memory thresholds</span>
<span class="nv">lower_threshold</span><span class="o">=</span><span class="k">$((</span><span class="nv">$MAX_MEMORY</span> <span class="o">*</span> <span class="nv">$LOWTHRES</span> <span class="o">/</span> <span class="m">100</span><span class="k">))</span>
<span class="nv">upper_threshold</span><span class="o">=</span><span class="k">$((</span><span class="nv">$MAX_MEMORY</span> <span class="o">*</span> <span class="nv">$UPTHRES</span> <span class="o">/</span> <span class="m">100</span><span class="k">))</span>
<span class="c"># Adjust memory allocation</span>
<span class="k">if</span> <span class="o">((</span>available_memory < lower_threshold<span class="o">))</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="nt">-n</span> <span class="s2">"increase "</span>
<span class="nv">new_memory</span><span class="o">=</span><span class="k">$((</span><span class="nv">$current_memory</span> <span class="o">+</span> <span class="o">(</span><span class="m">512</span> <span class="o">*</span> <span class="m">1024</span><span class="k">))</span><span class="o">)</span>
<span class="k">elif</span> <span class="o">((</span>available_memory <span class="o">></span> upper_threshold<span class="o">))</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="nt">-n</span> <span class="s2">"decrease "</span>
<span class="nv">new_memory</span><span class="o">=</span><span class="k">$((</span><span class="nv">$current_memory</span> <span class="o">-</span> <span class="o">(</span><span class="m">512</span> <span class="o">*</span> <span class="m">1024</span><span class="k">))</span><span class="o">)</span>
<span class="k">else
</span><span class="nb">echo</span> <span class="s2">"keeping current </span><span class="k">$((</span><span class="nv">$current_memory</span> <span class="o">/</span> <span class="m">1024</span> <span class="k">))</span><span class="s2">M avail: </span><span class="k">$((</span><span class="nv">$available_memory</span> <span class="o">/</span> <span class="m">1024</span><span class="k">))</span><span class="s2">M | lowt: </span><span class="k">$((</span><span class="nv">$lower_threshold</span> <span class="o">/</span> <span class="m">1024</span> <span class="k">))</span><span class="s2">M | upt: </span><span class="k">$((</span><span class="nv">$upper_threshold</span> <span class="o">/</span> <span class="m">1024</span> <span class="k">))</span><span class="s2">M"</span>
<span class="k">continue
fi</span>
<span class="c"># Ensure the new memory does not exceed the maximum</span>
<span class="k">if</span> <span class="o">((</span>new_memory <span class="o">></span> MAX_MEMORY<span class="o">))</span><span class="p">;</span> <span class="k">then
</span><span class="nv">new_memory</span><span class="o">=</span><span class="nv">$MAX_MEMORY</span>
<span class="k">fi</span>
<span class="c"># Ensure the new memory does not fall below the minimum</span>
<span class="k">if</span> <span class="o">[[</span> <span class="nv">$new_memory</span> < <span class="nv">$MIN_MEMORY</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
</span><span class="nv">new_memory</span><span class="o">=</span><span class="nv">$MIN_MEMORY</span>
<span class="k">fi
</span><span class="nb">echo</span> <span class="s2">"max: </span><span class="k">$((</span><span class="nv">$MAX_MEMORY</span> <span class="o">/</span> <span class="m">1024</span> <span class="k">))</span><span class="s2">M | cur: </span><span class="k">$((</span><span class="nv">$current_memory</span> <span class="o">/</span> <span class="m">1024</span> <span class="k">))</span><span class="s2">M | avail: </span><span class="k">$((</span><span class="nv">$available_memory</span> <span class="o">/</span> <span class="m">1024</span> <span class="k">))</span><span class="s2">M | lowt: </span><span class="k">$((</span><span class="nv">$lower_threshold</span> <span class="o">/</span> <span class="m">1024</span> <span class="k">))</span><span class="s2">M | upt: </span><span class="k">$((</span><span class="nv">$upper_threshold</span> <span class="o">/</span> <span class="m">1024</span> <span class="k">))</span><span class="s2">M => </span><span class="k">$((</span><span class="nv">$new_memory</span> <span class="o">/</span> <span class="m">1024</span> <span class="k">))</span><span class="s2">M"</span>
<span class="c"># Apply the new memory allocation</span>
virsh <span class="nt">--connect</span> <span class="s2">"</span><span class="nv">$uri</span><span class="s2">"</span> setmem <span class="s2">"</span><span class="nv">$vm_name</span><span class="s2">"</span> <span class="s2">"</span><span class="nv">$new_memory</span><span class="s2">"</span>
<span class="k">fi
done</span>
<span class="c"># echo "waiting for next execution"</span>
<span class="o">[[</span> <span class="s2">"</span><span class="nv">$daemon</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"1"</span> <span class="o">]]</span> <span class="o">&&</span> <span class="nb">sleep </span>7
<span class="k">do </span><span class="nb">true</span><span class="p">;</span> <span class="k">done</span>
</code></pre></div></div>
<p><a href="https://gist.github.com/stylefish/d337d787e9a643d3a8cd6b520aae0259">github gist</a></p>
<p>maybe this helps someone or at least me when reinstalling my machine ;)</p>
<p>greetings</p>angrydev blogwebmaster@angrydev.neton my primary machine that is currently using fedora i have a windows guest for some testing stuff that at some point needs more ram to do some tasks. to support that scenario i added a ballooning device and installed the qemu / virtio guest tools. at this point i’m able to manually adjust the memory allocation of the vm using the balloning driver using virsh:Missing title bar fix for rocket.chat (maybe other electron based apps) in fedora running gnome on wayland2023-04-18T10:32:00-05:002023-04-18T10:32:00-05:00http://blog.angrydev.net/rocket-chat-missing-title-bar-fix-on-wayland<p>During a recent “Rocket.Chat” update (happens for me since ~3.8.15) my title-bar was gone in gnome under wayland using fedora 37.
a quick search resulted in <a href="https://www.reddit.com/r/gnome/comments/yhweb7/gnome_electron_apps_in_native_wayland_mode_dont/">this</a> reddit post. there is suggested to add these features to the rocket.chat startup file:</p>
<p><code class="language-plaintext highlighter-rouge">--enable-features=UseOzonePlatform,WaylandWindowDecorations --ozone-platform=wayland</code></p>
<p>together with my earlier post to make the desktop share work we end up in this command line addition to the normal flatpak command line:</p>
<p><code class="language-plaintext highlighter-rouge">--enable-features=WebRTCPipeWireCapturer,UseOzonePlatform,WaylandWindowDecorations --ozone-platform=wayland</code></p>
<p>for convenience i copy / pasted my instructions for the complete setup process from my earlier post here (thank me later future me, that is copy pasting this into his freshly installed fedora):</p>
<p>find your currently used <code class="language-plaintext highlighter-rouge">.desktop</code> file for rocket chat. mine was at <code class="language-plaintext highlighter-rouge">/var/lib/flatpak/exports/share/applications/chat.rocket.RocketChat.desktop</code> and copy it to your users local applications dir:</p>
<p><code class="language-plaintext highlighter-rouge"># cp /var/lib/flatpak/exports/share/applications/chat.rocket.RocketChat.desktop ~/.local/share/applications</code></p>
<p>fire up an editor of your choice with the freshly created file (which takes precendence over the default one):</p>
<p><code class="language-plaintext highlighter-rouge"># nano ~/.local/share/applications/chat.rocket.RocketChat.desktop</code></p>
<p>and chat the executable line to:</p>
<p><code class="language-plaintext highlighter-rouge">Exec=/usr/bin/flatpak run --branch=stable --arch=x86_64 --file-forwarding chat.rocket.RocketChat @@u %U @@ --command=/app/bin/rocketchat-desktop --enable-features=WebRTCPipeWireCapturer,UseOzonePlatform,WaylandWindowDecorations --ozone-platform=wayland</code></p>angrydev blogwebmaster@angrydev.netDuring a recent “Rocket.Chat” update (happens for me since ~3.8.15) my title-bar was gone in gnome under wayland using fedora 37. a quick search resulted in this reddit post. there is suggested to add these features to the rocket.chat startup file:Using Komplete Audio 6 MK2 Input 3/4 on linux with pulse audio loopback for vinyl playback2022-10-29T07:19:00-05:002022-10-29T07:19:00-05:00http://blog.angrydev.net/using-komplete-audio-6-mk2-input-34-on-linux-with-pulse-audio-loopback-for-vinyl-playback<p>I have a Technics SL-1210 next to my PC and wanted to use my Native Instruments Komplete Audio 6 MK2’s back side inputs 3 and 4 (stereo)
to listen to vinyls while i’m working (“pass-through” or “loopback”) on my Manjaro Linux machine using pulseaudio.</p>
<p>the first problem: Only Inputs 1/2 are available in pulse audios <code class="language-plaintext highlighter-rouge">pavucontrol</code> in the tab “configuration” where you can set profiles for your card.
to view sinks (outputs) using the shell you could also use <code class="language-plaintext highlighter-rouge">pacmd list-sinks</code>.</p>
<p>so it seems the there is no configuration fo my usecase “Input 3/4 -> Output 1/2”…
Lets change that:</p>
<p>open <code class="language-plaintext highlighter-rouge">/usr/share/pulseaudio/alsa-mixer/profile-sets/native-instruments-komplete-audio6.conf</code> (on my manjaro machine)
or <code class="language-plaintext highlighter-rouge">/usr/share/alsa-card-profile/mixer/profile-sets</code> (for fedora)
and add our desired configuration:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[Profile output:analog-stereo-out-ab+input:analog-stereo-in-cd]
description = Analog Stereo Output 1/2, Analog Stereo Input 3/4
output-mappings = analog-stereo-out-ab
input-mappings = analog-stereo-in-cd
priority = 80
skip-probe = yes
</code></pre></div></div>
<p>i’ve used a profile and just changed the description and the input-mappings (from <code class="language-plaintext highlighter-rouge">ab</code> to <code class="language-plaintext highlighter-rouge">cd</code>).</p>
<p>now after a pulseaudio restart using <code class="language-plaintext highlighter-rouge">pulseaudio -k</code> i have “Input 3/4” available and “Output 1/2” when selecting the new profile in <code class="language-plaintext highlighter-rouge">pavucontrol</code>.</p>
<p>last thing to do is to “loopback” the input “3/4” directly to our output “1/2”, this is done using the following command:<br />
<code class="language-plaintext highlighter-rouge">pactl load-module module-loopback source=alsa_input.usb-Native_Instruments_Komplete_Audio_6_MK2_247C57F8-00.analog-stereo-in-cd sink=alsa_output.usb-Native_Instruments_Komplete_Audio_6_MK2_247C57F8-00.analog-stereo-out-ab</code><br />
<em>note: your device names might differ from mine, you can find your names using the following commands:</em></p>
<p>list sources (inputs):<br />
<code class="language-plaintext highlighter-rouge">pacmd list-sources | grep -E 'name:|index:'</code></p>
<p>list sinks (outputs):<br />
<code class="language-plaintext highlighter-rouge">pacmd list-sinks | grep -E 'name:|index:'</code></p>
<p>now you should hear the input from “Input 3/4” on your speakers connected to “Output 1/2”! :)</p>
<p>if you want your changes persistent you can either just copy the above command without <code class="language-plaintext highlighter-rouge">pactl</code> to <code class="language-plaintext highlighter-rouge">/etc/pulse/default.pa</code>
or you can created a custom <code class="language-plaintext highlighter-rouge">default.pa</code> in your home directory:</p>
<p>open or create <code class="language-plaintext highlighter-rouge">~/.config/pulse/default.pa</code> and put the following in:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>.include /etc/pulse/default.pa
load-module module-loopback source=alsa_input.usb-Native_Instruments_Komplete_Audio_6_MK2_247C57F8-00.analog-stereo-in-cd sink=alsa_output.usb-Native_Instruments_Komplete_Audio_6_MK2_247C57F8-00.analog-stereo-out-ab
</code></pre></div></div>
<p><em>note that if you changed your commandline earlier you have to make this changes here too, obviously</em></p>angrydev blogwebmaster@angrydev.netI have a Technics SL-1210 next to my PC and wanted to use my Native Instruments Komplete Audio 6 MK2’s back side inputs 3 and 4 (stereo) to listen to vinyls while i’m working (“pass-through” or “loopback”) on my Manjaro Linux machine using pulseaudio.Using Rocket.Chat (or any electron based app) screenshare on wayland2022-07-21T08:49:00-05:002022-07-21T08:49:00-05:00http://blog.angrydev.net/using-rocketchat-screen-share-with-wayland<p>When trying to use my company chat app “Rocket.Chat” i was struggling to get the desktop share to work. i’m using Rocket.Chat as a flatpak app on my manjaro machine.
when i select a desktop to share the screen was always just black. this did not happen on a X11 session, so i got stuck on x11 until this problem will be fixed by me!
today was the day…</p>
<p>a bit of googling brought me to the conclusion that i have to tell the electron app to use the <code class="language-plaintext highlighter-rouge">WebRTCPipeWireCapturer</code> feature.
here’s my fairly easy way to achieve this.</p>
<p>find your currently used <code class="language-plaintext highlighter-rouge">.desktop</code> file for rocket chat. mine was at <code class="language-plaintext highlighter-rouge">/var/lib/flatpak/exports/share/applications/chat.rocket.RocketChat.desktop</code> and copy it to your users local applications dir:</p>
<p><code class="language-plaintext highlighter-rouge"># cp /var/lib/flatpak/exports/share/applications/chat.rocket.RocketChat.desktop ~/.local/share/applications</code></p>
<p>fire up an editor of your choice with the freshly created file (which takes precendence over the default one):</p>
<p><code class="language-plaintext highlighter-rouge"># nano ~/.local/share/applications/chat.rocket.RocketChat.desktop</code></p>
<p>and chat the executable line to:</p>
<p><code class="language-plaintext highlighter-rouge">Exec=/usr/bin/flatpak run --branch=stable --arch=x86_64 --file-forwarding chat.rocket.RocketChat @@u %U @@ --command=/app/bin/rocketchat-desktop --enable-features=WebRTCPipeWireCapturer</code></p>
<p>my resulting file looks like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[Desktop Entry]
Name=Rocket.Chat
#Exec=/usr/bin/flatpak run --branch=stable --arch=x86_64 --command=/app/bin/rocketchat-desktop --file-forwarding chat.rocket.RocketChat @@u %U @@
Exec=/usr/bin/flatpak run --branch=stable --arch=x86_64 --file-forwarding chat.rocket.RocketChat @@u %U @@ --command=/app/bin/rocketchat-desktop --enable-features=WebRTCPipeWireCapturer
Terminal=false
Type=Application
Icon=chat.rocket.RocketChat
StartupWMClass=Rocket.Chat
MimeType=x-scheme-handler/rocketchat
Comment=Official OSX, Windows, and Linux Desktop Clients for Rocket.Chat
Categories=GNOME;GTK;Network;InstantMessaging;
X-Desktop-File-Install-Version=0.26
X-Flatpak-RenamedFrom=rocketchat-desktop.desktop;
X-Flatpak=chat.rocket.RocketChat
</code></pre></div></div>
<p>restart rocket.chat and you’re good to go. you will see a new switcher which lets you choose what to share after the original one from rocket.chat.</p>
<p>this should work for any electron app.</p>
<p>note that in my case the <code class="language-plaintext highlighter-rouge">--command</code> argument had to be the last one, otherwise my rocket.chat would not start…</p>angrydev blogwebmaster@angrydev.netWhen trying to use my company chat app “Rocket.Chat” i was struggling to get the desktop share to work. i’m using Rocket.Chat as a flatpak app on my manjaro machine. when i select a desktop to share the screen was always just black. this did not happen on a X11 session, so i got stuck on x11 until this problem will be fixed by me! today was the day…fixing Microsoft.DotNet.PlatformAbstractions not found on dotnet publish2021-07-14T09:39:00-05:002021-07-14T09:39:00-05:00http://blog.angrydev.net/fixing-microsoft-dotnet-platformabstractions-not-found<p>so today i (again) had to solve an issue with a new build agent with VS tools installed and trying to build a .net core application using <code class="language-plaintext highlighter-rouge">dotnet publish myapp.csproj -r win10-x64 --framework netcoreapp3.1 --self-contained=false</code>
wich would then fail with the following error. so i decided to blog this in case i have to solve it again ;)</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.DotNet.PlatformAbstractions, Version=2.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.
</code></pre></div></div>
<p>to resolve this i ran <code class="language-plaintext highlighter-rouge">dotnet --version</code> wich gave me a 5.x version and <code class="language-plaintext highlighter-rouge">dotnet --list-sdks</code> which gave me a 3.1 and a 5.0 sdk. should be OK right?
no… my current workaround is to move the “5.0” folder located in <code class="language-plaintext highlighter-rouge">Program Files\dotnet\sdk</code> somwhere else. i cannot uninstall this sdk because it came
with VS and i have no idea how to remove it in there, the uninstaller just says “please use VS installer to remove this product.”</p>
<p>works now, hopefully i do not need to cross-compile 3.1 and 5.x at some point! :D</p>angrydev blogwebmaster@angrydev.netso today i (again) had to solve an issue with a new build agent with VS tools installed and trying to build a .net core application using dotnet publish myapp.csproj -r win10-x64 --framework netcoreapp3.1 --self-contained=false wich would then fail with the following error. so i decided to blog this in case i have to solve it again ;)Microsoft.WebApplication.targets: How to install it using the command line2021-04-15T10:53:00-05:002021-04-15T10:53:00-05:00http://blog.angrydev.net/install-vs-build-agent-for-webapplications<h2 id="the-story">the story</h2>
<p>I have a running build built on an agent that is somehow quite aged, so i wanted to install a fresh one on a “windows server core” and script that with a bit of powershell.
After i’ve sone some setup an old and legacy webapp complains:</p>
<h2 id="the-error">the error</h2>
<p><code class="language-plaintext highlighter-rouge">error MSB4226: The imported project "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets" was not found. Also, tried to find "WebApplications\Microsoft.WebApplication.targets" in the fallback search path(s) for $(VSToolsPath) - "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v16.0" . These search paths are defined in "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\msbuild.exe.Config". Confirm that the path in the <Import> declaration is correct, and that the file exists on disk in one of the search paths.</code></p>
<p>so i looked and <code class="language-plaintext highlighter-rouge">Microsoft.WebApplication.targets</code> was indeed missing, although i had installed the “buildtools” workload <code class="language-plaintext highlighter-rouge">Microsoft.VisualStudio.Workload.WebBuildTools</code>.</p>
<p>just for reference (and me searching the internet finding my own blog):
another component id i put here for reference is: <code class="language-plaintext highlighter-rouge">Microsoft.VisualStudio.Component.Wcf.Tooling</code> when the file <code class="language-plaintext highlighter-rouge">Microsoft.VisualStudio.ServiceModel.targets</code> is missing :)</p>
<h2 id="some-research">some research</h2>
<p>ok, well ddg and google werent of much help (this is why i’m finally writing this).
Everybody was just suggesting to install VS using the installer, but they did not exactly point out what workload / workload ID i had to use.
so the question “How to install Microsoft.WebApplication.targets using a quiet, unattended Visual Studio setup?” arised.</p>
<p>i decided to check myself what workload it could be and tried some workloads i found on ms docs for <a href="https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2019">build tools component ids</a>, this is well documented, it couldn’t be that hard, right?</p>
<p>problem with all of them was: everything with “web” in it wouldnt install the <code class="language-plaintext highlighter-rouge">targets</code> file…</p>
<p>so i finally figured out i had to use a intaller of a “higher” edition of vs, like professional or enterprise, so i tried using the <code class="language-plaintext highlighter-rouge">vs_enterprise</code> installer and <a href="https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-enterprise?view=vs-2019">the component ids of vs enterprise</a></p>
<h2 id="the-fix">the fix</h2>
<p>finally this component id resulted in success: <code class="language-plaintext highlighter-rouge">Microsoft.VisualStudio.Component.Web</code>.</p>
<p>for an automated build agent setup use this command line:
<code class="language-plaintext highlighter-rouge">vs_enterprise.exe --quiet --add "Microsoft.VisualStudio.Component.Web"</code></p>
<h3 id="it-does-not-work-what-now">it does not work, what now?</h3>
<p>if this does not work and the installer exists shortly after you started it you may have to <a href="https://docs.microsoft.com/en-us/visualstudio/install/update-a-network-installation-of-visual-studio?view=vs-2019#deploy-an-update-to-client-machines">update</a> VS itself first (this is advisable generally). You can achieve that using this command:</p>
<p><code class="language-plaintext highlighter-rouge">C:\temp\buildPrerequisites\vs_enterprise.exe update --installPath "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise" --quiet --wait --norestart</code></p>
<p>after that (you can monitor if it finished in the task manager: see if all processes exited) restart the computer and try to install your missing component again.</p>angrydev blogwebmaster@angrydev.netthe storyHow to get remote audio microphone working with Windows Server 2019 Remote Desktop Services2019-08-19T18:00:00-05:002019-08-19T18:00:00-05:00http://blog.angrydev.net/how-to-get-remote-audio-microphone-working-with-windows-server-2019-remote-desktop-services<p>long story short, there were a few things that helped me when setting up remote services on a RDG / RDP server,
but the 2 main reasons it would not work (Remote Audio Microphone was present but it did not make any sounds).</p>
<p>On the Terminal Server as an admin:</p>
<p>Local Computer Policy (although this settings should be enabled by default if set to “Not specified”):</p>
<p>Administrative Templates -> Windows Components ->
Remote Desktop Services -> Remote Desktop Session Host -> Device and Resource Redirection:</p>
<p>Enable “Allow audio and video playback Redirection”
Enable “Allow audio recording redirection”</p>
<p>On the Terminal Server as the User:</p>
<p>Right Click “Sound” in the Taskbar (the Speaker Icon) ->
Open Sound Settings -> Microphone Privacy Settings</p>
<p>Enable “Allow apps to access your microphone”</p>
<p>With this i got remote mic up and running. :)</p>angrydev blogwebmaster@angrydev.netlong story short, there were a few things that helped me when setting up remote services on a RDG / RDP server, but the 2 main reasons it would not work (Remote Audio Microphone was present but it did not make any sounds). On the Terminal Server as an admin:a more complex proxy pac / wpad.dat example2017-10-24T18:00:00-05:002017-10-24T18:00:00-05:00http://blog.angrydev.net/a-more-complex-proxy-pac-wpad-dat-example<p>some of my time in the last few days i spent on a more complex wpad.dat or proxy pac file, that would support my
needs for a slightly more complex network structure than just a simple "use the proxy always, we only have
192.168.0.x", but with multiple networks where not all networks should use the proxy and some internal networks
should be accessed directly and some via proxy etc.</p>
<p>so i wrote the script below. it's pretty straight forward:</p>
<ul>
<li>proxyhost, proxyport: change the proxy host and port<br />[note: ${asg_hostname} is a variable used by Sophos UTM,
formerly known as astaro security gateway (asg) an is replaced with the hostname of the firewall/proxyserver]
</li>
<li>directRegexPatterns: add / change the networks you want to be accessed directly (without proxy)</li>
<li>nets: put the networks here that should use the proxy</li>
</ul>
<p>and thats it - throw it in your proxy server and fire it up!</p>
<p>good to know:</p>
<p>debugging in chrome is pretty easy, just add</p>
<pre>alert("my log info");</pre>
<p>to the proxy pac script and go to "<a href="chrome://net-internals/#proxy">chrome://net-internals/#proxy</a>"<br />there you see the proxy chrome is
using right now and can "re-apply" the settings (e.g. if you use proxy autoconfiguration via DHCP Option 252 [not
working in Firefox] or via DNS wpad.yourdomain.local [working with Firefox])</p>
<p>after you verified you're using the correct proxy you can go to "events" tab and sort by ID descending and search for
"PAC" and you'll see something like this:</p>
<p><a href="/assets/25-10-_2017_19-53-31.jpg"><img class="wp-image-212 alignnone" src="/assets/25-10-_2017_19-53-31.jpg" alt="" width="630" height="144" /></a></p>
<p>heres the gist of it. just download it and upload it to your firewall or other proxy server (maybe you have to rename
it to wpad.dat)</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">FindProxyForURL</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="nx">host</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// SETTINGS</span>
<span class="c1">// your proxy hostname (myproxy.mynetwork.local)</span>
<span class="kd">var</span> <span class="nx">proxyHostname</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">${asg_hostname}</span><span class="dl">"</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">proxyPort</span> <span class="o">=</span> <span class="mi">8080</span><span class="p">;</span>
<span class="c1">// regex patterns to exclude from proxy (put your internal networks here)</span>
<span class="kd">var</span> <span class="nx">directRegexPatterns</span> <span class="o">=</span> <span class="p">[</span>
<span class="dl">"</span><span class="s2">*.local/*</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">*.lan/*</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">*192.168.0.*</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">*172.16.*</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">*172.30.0.*</span><span class="dl">"</span>
<span class="p">];</span>
<span class="c1">// networks that should use proxies with optional proxy to use override (put your internal networks that should be proxied here)</span>
<span class="kd">var</span> <span class="nx">nets</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span> <span class="na">addr</span><span class="p">:</span> <span class="dl">"</span><span class="s2">172.16.0.0</span><span class="dl">"</span><span class="p">,</span> <span class="na">subnet</span><span class="p">:</span> <span class="dl">"</span><span class="s2">255.255.0.0</span><span class="dl">"</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">addr</span><span class="p">:</span> <span class="dl">"</span><span class="s2">172.30.0.0</span><span class="dl">"</span><span class="p">,</span> <span class="na">subnet</span><span class="p">:</span> <span class="dl">"</span><span class="s2">255.255.255.0</span><span class="dl">"</span><span class="p">,</span> <span class="na">proxy</span><span class="p">:</span> <span class="dl">"</span><span class="s2">172.30.0.1:</span><span class="dl">"</span> <span class="o">+</span> <span class="nx">proxyPort</span> <span class="p">},</span>
<span class="p">{</span> <span class="na">addr</span><span class="p">:</span> <span class="dl">"</span><span class="s2">192.168.0.0</span><span class="dl">"</span><span class="p">,</span> <span class="na">subnet</span><span class="p">:</span> <span class="dl">"</span><span class="s2">255.255.255.0</span><span class="dl">"</span> <span class="p">}</span>
<span class="p">];</span>
<span class="kd">var</span> <span class="nx">p</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">DIRECT</span><span class="dl">"</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">defaultproxyurl</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">PROXY </span><span class="dl">"</span> <span class="o">+</span> <span class="nx">proxyHostname</span> <span class="o">+</span> <span class="dl">"</span><span class="s2">:</span><span class="dl">"</span> <span class="o">+</span> <span class="nx">proxyPort</span><span class="p">;</span>
<span class="c1">// ACTIONS</span>
<span class="c1">//Don't proxy connections to the proxy web interface</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">shExpMatch</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="dl">"</span><span class="s2">https://${asg_hostname}*</span><span class="dl">"</span><span class="p">))</span> <span class="p">{</span> <span class="nx">p</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">DIRECT</span><span class="dl">"</span><span class="p">;</span> <span class="p">}</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">shExpMatch</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="dl">"</span><span class="s2">https://</span><span class="dl">"</span> <span class="o">+</span> <span class="nx">dnsResolve</span><span class="p">(</span><span class="nx">host</span><span class="p">)</span> <span class="o">+</span> <span class="dl">"</span><span class="s2">*</span><span class="dl">"</span><span class="p">))</span> <span class="p">{</span> <span class="nx">p</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">DIRECT</span><span class="dl">"</span><span class="p">;</span> <span class="p">}</span>
<span class="c1">//Exclude non-fqdn hosts from being proxied</span>
<span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="nx">isPlainHostName</span><span class="p">(</span><span class="nx">host</span><span class="p">))</span> <span class="p">{</span> <span class="nx">p</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">DIRECT</span><span class="dl">"</span><span class="p">;</span> <span class="p">}</span>
<span class="k">else</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">hasRegexMatch</span> <span class="o">=</span> <span class="kc">false</span><span class="p">;</span>
<span class="c1">// check proxy exclusion regex patterns</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">var</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">directRegexPatterns</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">pattern</span> <span class="o">=</span> <span class="nx">directRegexPatterns</span><span class="p">[</span><span class="nx">i</span><span class="p">];</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">shExpMatch</span><span class="p">(</span><span class="nx">url</span><span class="p">,</span> <span class="nx">pattern</span><span class="p">))</span> <span class="p">{</span>
<span class="nx">p</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">DIRECT</span><span class="dl">"</span><span class="p">;</span>
<span class="nx">hasRegexMatch</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="k">break</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">hasRegexMatch</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// check if client is in proxy network</span>
<span class="kd">var</span> <span class="nx">ipstr</span> <span class="o">=</span> <span class="dl">""</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="k">typeof</span> <span class="nx">myIpAddressEx</span> <span class="o">===</span> <span class="dl">"</span><span class="s2">undefined</span><span class="dl">"</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">alert</span><span class="p">(</span><span class="dl">"</span><span class="s2">myIpAddressEx is undefined!</span><span class="dl">"</span><span class="p">);</span> <span class="c1">// this will print to a specific log in the browser</span>
<span class="nx">ipstr</span> <span class="o">=</span> <span class="nx">myIpAddress</span><span class="p">();</span> <span class="c1">// only one ip, not all, but FF does not support the "Ex" version...</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="nx">ipstr</span> <span class="o">=</span> <span class="nx">myIpAddressEx</span><span class="p">();</span> <span class="c1">// IP1;IP2;IP3</span>
<span class="p">}</span>
<span class="kd">var</span> <span class="nx">ips</span> <span class="o">=</span> <span class="nx">ipstr</span><span class="p">.</span><span class="nx">split</span><span class="p">(</span><span class="dl">"</span><span class="s2">;</span><span class="dl">"</span><span class="p">);</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">var</span> <span class="nx">j</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">j</span> <span class="o"><</span> <span class="nx">nets</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">j</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">net</span> <span class="o">=</span> <span class="nx">nets</span><span class="p">[</span><span class="nx">j</span><span class="p">];</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">var</span> <span class="nx">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">ips</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">ip</span> <span class="o">=</span> <span class="nx">ips</span><span class="p">[</span><span class="nx">i</span><span class="p">];</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">isInNet</span><span class="p">(</span><span class="nx">ip</span><span class="p">,</span> <span class="nx">net</span><span class="p">.</span><span class="nx">addr</span><span class="p">,</span> <span class="nx">net</span><span class="p">.</span><span class="nx">subnet</span><span class="p">))</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">proxyToUse</span> <span class="o">=</span> <span class="nx">defaultproxyurl</span><span class="p">;</span>
<span class="k">if</span><span class="p">(</span><span class="nx">net</span><span class="p">.</span><span class="nx">proxy</span><span class="p">){</span>
<span class="nx">proxyToUse</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">PROXY </span><span class="dl">"</span> <span class="o">+</span> <span class="nx">net</span><span class="p">.</span><span class="nx">proxy</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">p</span> <span class="o">=</span> <span class="nx">proxyToUse</span><span class="p">;</span>
<span class="c1">// alert("found " + proxyToUse + " because: " + ip + " is in net " + net.addr + " / " + net.subnet);</span>
<span class="k">break</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nx">p</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p><a href="https://gist.github.com/stylefish/fb30747cae1b9b935e89cadc759820a1">github gist</a></p>angrydev blogwebmaster@angrydev.netsome of my time in the last few days i spent on a more complex wpad.dat or proxy pac file, that would support my needs for a slightly more complex network structure than just a simple "use the proxy always, we only have 192.168.0.x", but with multiple networks where not all networks should use the proxy and some internal networks should be accessed directly and some via proxy etc. so i wrote the script below. it's pretty straight forward: proxyhost, proxyport: change the proxy host and port[note: ${asg_hostname} is a variable used by Sophos UTM, formerly known as astaro security gateway (asg) an is replaced with the hostname of the firewall/proxyserver] directRegexPatterns: add / change the networks you want to be accessed directly (without proxy) nets: put the networks here that should use the proxy and thats it - throw it in your proxy server and fire it up! good to know: debugging in chrome is pretty easy, just add alert("my log info"); to the proxy pac script and go to "chrome://net-internals/#proxy"there you see the proxy chrome is using right now and can "re-apply" the settings (e.g. if you use proxy autoconfiguration via DHCP Option 252 [not working in Firefox] or via DNS wpad.yourdomain.local [working with Firefox]) after you verified you're using the correct proxy you can go to "events" tab and sort by ID descending and search for "PAC" and you'll see something like this: heres the gist of it. just download it and upload it to your firewall or other proxy server (maybe you have to rename it to wpad.dat)How to fix DBCC CHECKDB inconsistency errors in MS SQL Server caused by upgrading from ntext to nvarcharmax2017-07-09T18:00:00-05:002017-07-09T18:00:00-05:00http://blog.angrydev.net/how-to-fix-dbcc-checkdb-inconsistency-errors-in-ms-sql-server-caused-by-upgrading-from-ntext-to-nvarcharmax<h1>TL;DR</h1>we were facing database inconsistency issues detected by DBCC CHECKDB / DBCC CHECKTABLE that we're running
in nightly jobs. The cause of this was a as far as we've investigated a change from "ntext" / "text" column types to
"nvarchar(max)" / "varchar(max)", and an optimization to a component that stores large strings. this resulted in smaller
nvarchar(max) values and somehow caused the inconsistencies.dont worry, we've tested and found a solution that does not
corrupt your data, but nothing guaranteed, you do this on your own risk!<ol>
<li>take a backup, just in case</li>
<li>remove or disable constraints</li>
<li>rename the old table (e.g. MyTable -> MyTable_TEMP)</li>
<li>create the table structure again directly with nvarchar(max) column types</li>
<li>insert the complete data of the old temp table into the new one</li>
<li>drop the old temp table</li>
<li>restore the constraints.</li>
</ol>we've since then never faced this issue again.to learn more, read the long version below ;)<h1>the complete story
</h1>some day i had some alarming emails in my inbox from our sql server. somehow some "offrow data" was incorrecly
referenced.Error Message:
<pre>
Table error: Object ID 1234567, index ID 1, partition ID 987654321, alloc unit ID 123123123123 (type LOB data).
The off-row data node at page (1:12345678), slot 0, text ID 7766554433 does not match its reference from page (1:978563412), slot 1.
</pre>
after some googling i found some posts suggesting me to use "DBCC CHECKDB('MyDB', REPAIR_ALLOW_DATA_LOSS). yeah great
start of the day... "allow data loss" sound really great :D i will not go into details on how to fix these errors,
googling dbcc checkdb should be enough to find infos on that topic, though i was not out final solution. we've come up
with a "trick" to maintain all data and fix the corruption without data loss, but more on that later (or in
TL;DR).everything on the internet on how that happened blamed the storage subsystem. HDD broken, bit rot, etc. but we're
using ZFS as a storage subsystem and it would have telled me (weekly scrub) if there were any data losses etc. checked
the logs, but found noting that pointed to a storage issue. ok. i resolved the issue correcting the database with the
dbcc tool, this is strange, but i dont have much time now to chase this issue. it's resolved, so what. only happend
once, right?no... certainly not right. next week it happened again...<h2>the first try and the MS Support incident</h2>i
noticed the second time the error happened, that i got an error message from sql server of "severity 20" some time the
day before (or 2 days before) saying:
<pre>
DESCRIPTION: A read operation on a large object failed while sending data to the client.
A common cause for this is if the application is running in READ UNCOMMITTED isolation level. This connection will be terminated.
</pre>
so we began to investigate further knowing we're using this isolation level in our software the database belongs to.long
story short, we've investigated, talked to MS Support, they said ReadUncommitted is the issue, we changed out codebase
to not use ReadUncommitted anymore (instead we used snapshot as adviced by MS), weeks pass by, and... oh look the same
inconsistency error. contacted MS Support again, they said, they cannot help us, we have to open up a root case analysis
support case, that IMHO costs too much money. so back to the beginning. :(<h2>investigating on our own</h2>in the
meantime some of our customers contacted us, they also have inconsistencies and are worried about their data. so back at
the beginning, i thought what we have changed the past 3 versions in our software since this problem first
appeared.after some research i found these 2 points:<ol>
<li>We've improved how to store large strings, they now consume much less space in the ntext / nvarchar(max) field
of our database [due to compression]</li>
<li>We've changed the database layout of our tables to move away from "ntext" and instead use "nvarchar(max)" as <a
href="https://docs.microsoft.com/en-us/sql/t-sql/data-types/ntext-text-and-image-transact-sql">suggested by
MS.</a></li>
</ol>so what if we saved a large string when the column type was ntext and then updated the field with our new logic,
that compresses the content and save it back to the field that now has the column type nvarchar(max) and something with
the linking went wrong internally in SQL Server? Remember:
<pre>The off-row data node at page [...] does not match its reference from page [...]</pre>hmmm, so i decided to verify
my claim.i played around a little bit with <a
href="https://blogs.msdn.microsoft.com/sqlserverstorageengine/2006/06/10/how-to-use-dbcc-page/">DBCC PAGE</a>, to
look at pages and their content in sql server, and found the row that was reported as currupted by DBCC CHECKDB:
<pre>
Table error: Object ID 1234567, index ID 1, partition ID 987654321, alloc unit ID 123123123123 (type LOB data).
The off-row data node at page (1:12345678), slot 0, text ID 7766554433 does not match its reference from page <span style="color: #ff0000;">(1:978563412), slot 1</span>.
</pre>
ok, so the off row data (data that was pushed off the row because it was too big) has a pointer back to the row it
belongs to, and this pointer was referencing something wrong. to i looked at the referenced page (marked red above):
<pre>dbcc traceon(3604)dbcc page(MyDB, 1, <span style="color: #ff0000;">978563412, 1</span>) with tableresults</pre>and
got the primary key of the row that was affected. ok, now we've decyphered the message and are able to play around with
this data! yay!<h2>playing around until a solution emerges</h2>i tried to change the nvarchar(max) field data to the
same content via an update statement, no changes. ive copied the row: the new row was clean, no new inconsistencies. hm.
ok, so i deleted that row. inconsistency gone. yeah, thats what my friend DBCC CHECKDB Repair Allow Data Loss does, but
we dont want that. we want to keep our data. luckily i have some more rows that are affected. so on to the next row!but
this time i copied the row out to a temporary table, deleted it and inserted it back into its original table. tadaaa,
consistency restored, but without data loss. this is the solution we came up with. in general we wrote a script that
copies the complete table to another table, drop the old one and renamed the new one. since then we've not faced an
inconsistency in this table again. so we think it has to be related to updating the columns from ntext to
navarchar(max).these are the steps our "repair" script does for the table we've faced the issue. i know this solution is
far from perfect. your partly have doubled the data for this table (copy, rename and then drop) but its the only
solution we've come up with.<ol>
<li>take a backup, just in case</li>
<li>remove or disable constraints</li>
<li>rename the old table (e.g. MyTable -> MyTable_TEMP)</li>
<li>create the table structure again directly with nvarchar(max) column types</li>
<li>insert the complete data of the old temp table into the new one</li>
<li>drop the old temp table</li>
<li>restore the constraints.</li>
</ol>
<p>finally here are some sql statements i found useful, maybe they're helpful to somebody. :)</p>
<p>
<em>
<strong>all these
scipts may be used for any purpose, but at your own risk! nothing guaranteed. DBCC PAGE is an undocumented
feature, so be careful, take a backup before and try to test this stuff on a temporary/testing database
before
you go ahead and try to fix this on a production database!
</strong>
</em>
</p>
check contents of page and get primary
key of affected row:<div>
<pre>dbcc traceon(3604)dbcc page(<span style="color: #ff0000;">MyDB</span>, 1, <span style="color: #ff0000;">PageId</span>, <span style="color: #ff0000;">Slot</span>) with tableresults</pre>
</div>
<div>
<div>find all inconsistent rows and get their primary key
<pre>CREATE TABLE #dbcc_tableresults(
[Error] int,
[Level] int,
[State] int,
[MessageText] varchar(7000),
[RepairLevel] varchar(7000),
[Status] int,
[DbId] int,
[DbFragId] int,
[ObjectID] int,
[IndexId] int,
[PartitionId] bigint,
[AllocUnitId] bigint,
[RidDbId] int,
[RidPruId] int,
[File] int,
[Page] int,
[Slot] int,
[RefDbId] int,
[RefPruId] int,
[RefFile] int,
[RefPage] int,
[RefSlot] int,
[Allocation] int);
-- Execute CheckDB and insert result to temp table
INSERT INTO #dbcc_tableresults(
[Error],
[Level],
[State],
[MessageText],
[RepairLevel],
[Status],
[DbId],
[DbFragId],
[ObjectID],
[IndexId],
[PartitionId],
[AllocUnitId],
[RidDbId],
[RidPruId],
[File],
[Page],
[Slot],
[RefPruId],
[RefDbId],
[RefFile],
[RefPage],
[RefSlot],
[Allocation])
EXEC ('DBCC CHECKTABLE(<span style="color: #ff0000;">MyCurruptTableName</span>) WITH TABLERESULTS')</pre>
</div>
</div>
<div>with the above script you are able to create a script that only fixes your inconsistent rows, but in our case it
was just a matter of time until the next inconsistency occurs, so we've decided to copy the whole table.</div>
<div>i've omitted the script that copies the rows and re-inserts them because its heavily tied to our DB-Schema and
wouldn't be of great use to others.</div>
<div>maybe this helps other people in my position to find a solution sooner than i did... :)</div>angrydev blogwebmaster@angrydev.netTL;DRwe were facing database inconsistency issues detected by DBCC CHECKDB / DBCC CHECKTABLE that we're running in nightly jobs. The cause of this was a as far as we've investigated a change from "ntext" / "text" column types to "nvarchar(max)" / "varchar(max)", and an optimization to a component that stores large strings. this resulted in smaller nvarchar(max) values and somehow caused the inconsistencies.dont worry, we've tested and found a solution that does not corrupt your data, but nothing guaranteed, you do this on your own risk! take a backup, just in case remove or disable constraints rename the old table (e.g. MyTable -> MyTable_TEMP) create the table structure again directly with nvarchar(max) column types insert the complete data of the old temp table into the new one drop the old temp table restore the constraints. we've since then never faced this issue again.to learn more, read the long version below ;)the complete story some day i had some alarming emails in my inbox from our sql server. somehow some "offrow data" was incorrecly referenced.Error Message: Table error: Object ID 1234567, index ID 1, partition ID 987654321, alloc unit ID 123123123123 (type LOB data). The off-row data node at page (1:12345678), slot 0, text ID 7766554433 does not match its reference from page (1:978563412), slot 1. after some googling i found some posts suggesting me to use "DBCC CHECKDB('MyDB', REPAIR_ALLOW_DATA_LOSS). yeah great start of the day... "allow data loss" sound really great :D i will not go into details on how to fix these errors, googling dbcc checkdb should be enough to find infos on that topic, though i was not out final solution. we've come up with a "trick" to maintain all data and fix the corruption without data loss, but more on that later (or in TL;DR).everything on the internet on how that happened blamed the storage subsystem. HDD broken, bit rot, etc. but we're using ZFS as a storage subsystem and it would have telled me (weekly scrub) if there were any data losses etc. checked the logs, but found noting that pointed to a storage issue. ok. i resolved the issue correcting the database with the dbcc tool, this is strange, but i dont have much time now to chase this issue. it's resolved, so what. only happend once, right?no... certainly not right. next week it happened again...the first try and the MS Support incidenti noticed the second time the error happened, that i got an error message from sql server of "severity 20" some time the day before (or 2 days before) saying: DESCRIPTION: A read operation on a large object failed while sending data to the client. A common cause for this is if the application is running in READ UNCOMMITTED isolation level. This connection will be terminated. so we began to investigate further knowing we're using this isolation level in our software the database belongs to.long story short, we've investigated, talked to MS Support, they said ReadUncommitted is the issue, we changed out codebase to not use ReadUncommitted anymore (instead we used snapshot as adviced by MS), weeks pass by, and... oh look the same inconsistency error. contacted MS Support again, they said, they cannot help us, we have to open up a root case analysis support case, that IMHO costs too much money. so back to the beginning. :(investigating on our ownin the meantime some of our customers contacted us, they also have inconsistencies and are worried about their data. so back at the beginning, i thought what we have changed the past 3 versions in our software since this problem first appeared.after some research i found these 2 points: We've improved how to store large strings, they now consume much less space in the ntext / nvarchar(max) field of our database [due to compression] We've changed the database layout of our tables to move away from "ntext" and instead use "nvarchar(max)" as suggested by MS. so what if we saved a large string when the column type was ntext and then updated the field with our new logic, that compresses the content and save it back to the field that now has the column type nvarchar(max) and something with the linking went wrong internally in SQL Server? Remember: The off-row data node at page [...] does not match its reference from page [...]hmmm, so i decided to verify my claim.i played around a little bit with DBCC PAGE, to look at pages and their content in sql server, and found the row that was reported as currupted by DBCC CHECKDB: Table error: Object ID 1234567, index ID 1, partition ID 987654321, alloc unit ID 123123123123 (type LOB data). The off-row data node at page (1:12345678), slot 0, text ID 7766554433 does not match its reference from page (1:978563412), slot 1. ok, so the off row data (data that was pushed off the row because it was too big) has a pointer back to the row it belongs to, and this pointer was referencing something wrong. to i looked at the referenced page (marked red above): dbcc traceon(3604)dbcc page(MyDB, 1, 978563412, 1) with tableresultsand got the primary key of the row that was affected. ok, now we've decyphered the message and are able to play around with this data! yay!playing around until a solution emergesi tried to change the nvarchar(max) field data to the same content via an update statement, no changes. ive copied the row: the new row was clean, no new inconsistencies. hm. ok, so i deleted that row. inconsistency gone. yeah, thats what my friend DBCC CHECKDB Repair Allow Data Loss does, but we dont want that. we want to keep our data. luckily i have some more rows that are affected. so on to the next row!but this time i copied the row out to a temporary table, deleted it and inserted it back into its original table. tadaaa, consistency restored, but without data loss. this is the solution we came up with. in general we wrote a script that copies the complete table to another table, drop the old one and renamed the new one. since then we've not faced an inconsistency in this table again. so we think it has to be related to updating the columns from ntext to navarchar(max).these are the steps our "repair" script does for the table we've faced the issue. i know this solution is far from perfect. your partly have doubled the data for this table (copy, rename and then drop) but its the only solution we've come up with. take a backup, just in case remove or disable constraints rename the old table (e.g. MyTable -> MyTable_TEMP) create the table structure again directly with nvarchar(max) column types insert the complete data of the old temp table into the new one drop the old temp table restore the constraints. finally here are some sql statements i found useful, maybe they're helpful to somebody. :) all these scipts may be used for any purpose, but at your own risk! nothing guaranteed. DBCC PAGE is an undocumented feature, so be careful, take a backup before and try to test this stuff on a temporary/testing database before you go ahead and try to fix this on a production database! check contents of page and get primary key of affected row: dbcc traceon(3604)dbcc page(MyDB, 1, PageId, Slot) with tableresults find all inconsistent rows and get their primary key CREATE TABLE #dbcc_tableresults( [Error] int, [Level] int, [State] int, [MessageText] varchar(7000), [RepairLevel] varchar(7000), [Status] int, [DbId] int, [DbFragId] int, [ObjectID] int, [IndexId] int, [PartitionId] bigint, [AllocUnitId] bigint, [RidDbId] int, [RidPruId] int, [File] int, [Page] int, [Slot] int, [RefDbId] int, [RefPruId] int, [RefFile] int, [RefPage] int, [RefSlot] int, [Allocation] int); -- Execute CheckDB and insert result to temp table INSERT INTO #dbcc_tableresults( [Error], [Level], [State], [MessageText], [RepairLevel], [Status], [DbId], [DbFragId], [ObjectID], [IndexId], [PartitionId], [AllocUnitId], [RidDbId], [RidPruId], [File], [Page], [Slot], [RefPruId], [RefDbId], [RefFile], [RefPage], [RefSlot], [Allocation]) EXEC ('DBCC CHECKTABLE(MyCurruptTableName) WITH TABLERESULTS') with the above script you are able to create a script that only fixes your inconsistent rows, but in our case it was just a matter of time until the next inconsistency occurs, so we've decided to copy the whole table. i've omitted the script that copies the rows and re-inserts them because its heavily tied to our DB-Schema and wouldn't be of great use to others. maybe this helps other people in my position to find a solution sooner than i did... :)IsolationLevel ReadUncommitted using SqlConnection and TransactionScopes done right in c#2017-02-09T17:00:00-06:002017-02-09T17:00:00-06:00http://blog.angrydev.net/transactionscope-isolationlevel-readuncommitted-using-sqlconnection-and-transactionscopes-right-in-c<p>first, lets look at some code we’re using:</p>
<figure class="highlight"><pre><code class="language-csharp" data-lang="csharp"><span class="kt">var</span> <span class="n">options</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">TransactionOptions</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">IsolationLevel</span> <span class="p">=</span> <span class="n">IsolationLevel</span><span class="p">.</span><span class="n">ReadUncommitted</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">using</span><span class="p">(</span><span class="kt">var</span> <span class="n">ts</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">TransactionScope</span><span class="p">(</span><span class="n">TransactionScopeOption</span><span class="p">.</span><span class="n">RequiresNew</span><span class="p">,</span><span class="n">options</span><span class="p">))</span>
<span class="p">{</span>
<span class="k">using</span><span class="p">(</span><span class="kt">var</span> <span class="n">c</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">SqlConnection</span><span class="p">(...))</span>
<span class="p">{</span>
<span class="c1">// write some data with sql connection</span>
<span class="p">}</span>
<span class="p">}</span></code></pre></figure>
<p>note the “ReadUncommitted” transaction level above.if we look at the transaction level inside the using of the SqlConnection its,
like expected, “ReadUncommitted”.ok, so far so good, everything fine here. now, lets read some data:</p>
<figure class="highlight"><pre><code class="language-csharp" data-lang="csharp"><span class="k">using</span><span class="p">(</span><span class="kt">var</span> <span class="n">c</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">SqlConnection</span><span class="p">(...))</span>
<span class="p">{</span>
<span class="c1">// read some data with sql connection</span>
<span class="p">}</span></code></pre></figure>
<p>lets look at the transaction level again within the using: its “Read <strong>Uncommitted</strong>”.
hmm. but the <a href="https://msdn.microsoft.com/en-us/library/ms175909.aspx">default</a> level of SQL Server is “Read <strong>Committed</strong>” isnt it?!
lets rerun our program and execute this snippet again wihout executing a transaction scope before:</p>
<figure class="highlight"><pre><code class="language-csharp" data-lang="csharp"><span class="k">using</span><span class="p">(</span><span class="kt">var</span> <span class="n">c</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">SqlConnection</span><span class="p">(...))</span>
<span class="p">{</span>
<span class="c1">// read some data with sql connection</span>
<span class="p">}</span></code></pre></figure>
<p>“Read <strong>Committed</strong>“.to understand this behavior we need some facts:</p>
<ul>
<li>the <a href="https://msdn.microsoft.com/en-us/library/ms175909.aspx">default</a> transaction level for a new sql server session (connection) is “Read Committed”</li>
<li>the last setted transaction level is kept for the time the session exists (logical, if we set it, we expect it to be set until we change it)</li>
<li>SqlConnection is pooled by <a href="https://msdn.microsoft.com/en-us/library/8xx3tyca(v=vs.110).aspx">default</a> (can be disabled, but we want the speed improvements)</li>
</ul>
<p>ok, knowing all that stuff we learned above this is also comprehensible, but not really intuitive. i would have
expected the connection is reset when put back into the pool to its initial transaction level, whatever it was at
that time, but thats not the case.ok, knowing this behavior we still dont know how this could possibly cause a
database inconsistency…we contacted MS Support and wanted help on that topic. sent them the logs of this machine, they
said: you’re using “Read Uncommitted”, dont use that, use snapshot (available <a href="https://msdn.microsoft.com/en-us/library/ms173763.aspx">since sql 2008</a>).
because when using Read Uncommitted when <strong>reading</strong> data from a table with ntext or nvarchar(max) [this is our “off row data”
from the message above] this could lead to inconsistency.
so we ended up changing our code to <strong>ensure</strong> a transaction level before we’re using it. therefore we changed our ConnectionFactory (fortunately we had one ;)) to
execute something like this before returning a SqlConnection:</p>
<figure class="highlight"><pre><code class="language-csharp" data-lang="csharp"><span class="k">private</span> <span class="k">void</span> <span class="nf">EnsureDefaultIsolationLevel</span><span class="p">(</span><span class="n">DbConnection</span> <span class="n">connection</span><span class="p">,</span> <span class="n">IsolationLevel</span> <span class="n">isolationLevel</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">System</span><span class="p">.</span><span class="n">Transactions</span><span class="p">.</span><span class="n">Transaction</span><span class="p">.</span><span class="n">Current</span> <span class="p">!=</span> <span class="k">null</span><span class="p">)</span>
<span class="k">return</span><span class="p">;</span>
<span class="n">connection</span><span class="p">.</span><span class="nf">EnsureOpen</span><span class="p">();</span>
<span class="n">DbCommand</span> <span class="n">command</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(!</span><span class="nf">TryCreateCommand</span><span class="p">(</span><span class="k">out</span> <span class="n">command</span><span class="p">))</span>
<span class="k">return</span><span class="p">;</span>
<span class="k">using</span> <span class="p">(</span><span class="n">command</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">command</span><span class="p">.</span><span class="n">Connection</span> <span class="p">=</span> <span class="n">connection</span><span class="p">;</span>
<span class="kt">var</span> <span class="n">sqlIsolationLevel</span> <span class="p">=</span> <span class="n">IsolationLevelMap</span><span class="p">[</span><span class="n">isolationLevel</span><span class="p">];</span>
<span class="n">command</span><span class="p">.</span><span class="n">CommandText</span> <span class="p">=</span> <span class="s">$"SET TRANSACTION ISOLATION LEVEL </span><span class="p">{</span><span class="n">sqlIsolationLevel</span><span class="p">}</span><span class="s">;"</span><span class="p">;</span>
<span class="n">command</span><span class="p">.</span><span class="nf">ExecuteNonQuery</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span></code></pre></figure>
<p>so what this code does is simply: check if there is a transaction scope defined, if yes, do nothing because transactionscope defines a isolation level.
otherwise simply execute a “set transaction isolation level” command to ensure we’re not “recycling” the last isolation level.
good to know: why does first check “Transaction.Current != null” work? because when you dont define the transactionscope before
the connection is opened, the transactionscope is useless and the default transactionlevel is kept and no transaction is
created on the sql server.
until now the error has not happened again, so we (and the MS support guy) think this is the
solution. we’ll see ;)</p>
<p>hope this helps somebody to better understand how this stuff works internally and what the caveats
are you have to implement when you want to be sure which transaction level your sql connection uses if you don’t
explicitly define one - or maybe out there is somebody thats facing the exact same issue right now.</p>angrydev blogwebmaster@angrydev.netfirst, lets look at some code we’re using: