Really useful Nuget packages: FastMember. Convert a generic list to a DbDataReader on the quick.

Thursday, October 5, 2017 by Nate Bross

Utilizing .NET Core has been a pretty great experience. There have been a few gotchas with APIs not being available in the base package. I was really stoked to see that the SqlBulkCopy classes are part of .NET Core. I was less thrilled to note that DataTable is there in .NET Core 1.0 but just an empty non-usable class. 

That means converting from a generic IEnumerable<T> to a DataTable/Set is not an option.

Enter DbDataReader: another way to utilize BulkCopy.

If you have an IDataReader instance, the BulkCopy WriteToServer method has an overload to cover that; however, I'm an ORM to pull in some data form various sources so I basically have List<T>s, not IDataReaders. Searching the web it's pretty difficult to find a generic way to convert from a generic collection to a IDataReader. Much harder than it should be.

Enter FastMember: Convert an IEnumerable<T> to a DbDataReader, fast!

This great package makes the process easy and extremely fast. Basic demo shows how simple this package makes things. 

using (SqlBulkCopy bulkcopy = new SqlBulkCopy(connection)
{
    using (var reader = ObjectReader.Create(toInsert))
    {
        bulkcopy.WriteToServer(reader);
    }
    bulkcopy.Close();
}

Anatomy of a blob storage Uri, and how to use a blob name prefix to make Azure do your filtering.

Wednesday, July 5, 2017 by Nate Bross

Sounds simple enough, right? The blob storage account has a Uri and each part means something.

There are only three levels of hierarchy built into the system:

  1. Account
  2. Container
  3. Blob

Seen as:

https://[STORAGE_ACCOUNT_NAME].blob.core.windows.net/[CONTAINER-NAME]/[BLOB-NAME]

Within the Blob itself, the NAME property can be used to create additional ‘virtual’ directories, but they are just that. Virtual. This is where things get pretty powerful. Using the Storage Client libraries for .NET, the ListBlobsSegmentedAsync method allows you to have Azure filter out blobs based on prefix. The prefix filter here only applies to the Blob Name. If we look at a specific example (redacted to protect the guilty):

https://[STORAGE_ACCOUNT].blob.core.windows.net/[CONTAINER]/VirtualFolder1/2017/7/01/15/fileanme.ext

You see this whole part [VirtualFolder1/2017/07/01/15/fileanme.ext] is all the Blob Name. It just so happens to be setup by folders Year/Month/Date/Hour and because of this we can use the ListBlobsSegmentedAsync to filter based on it.

var list = await container.ListBlobsSegmentedAsync(
"VirtualFolder1/2017/07",
true,
BlobListingDetails.None,
int.MaxValue,
null,
null,
null);

This would give us all files for the month of July in the year 2017, regardless of which day or hour they are listed in. Doing it this way allows me to query a much more limited set of data, meaning I have to process out fewer files locally and there is less data transfer in and out as a result.

The main caveat I’ve found is that you cannot use wildcards, so you can’t find all the blobs for July in any year without doing multiple queries. Because the container is not part of the blob name, you cannot query across containers either.

Error Migrating App Service Plan because of ‘microsoft.insights/webtests’

Tuesday, May 23, 2017 by Nate Bross

Had two azure subscriptions along with two AppServicePlans (one each) for cost savings I wanted to combine them, but you can’t use the ‘Change AppServicePlan’ on the individual AppServices unless they’re in the same sub, resource group, and region.

Each time I tried to use the portal to move one AppServicePlan to the other subscription, I would get this error:

the subscription ‘subscription-guid’ is not registered for resource types 'microsoft.insights/webtests (centralus)'. #code: missingregistrationsfortypes#

Turns out that I had some existing web tests from the old portal that were orphaned and couldn’t be opened/read from the new portal.

Enter the commandline tool from the web portal (yes, the terminal in the web portal, its neat check it out!):

az resource list –resource-type microsoft.insights/webtests

and there I get a nice json list of the resources that were giving me trouble, along with their “id” so I was able to delete them with

az resource delete –id /subscriptions/[guid]/resourceGroups/[my-resource-group]/providers/microsoft.insights.webtests/[test-name]

Someone more in tune with the bash shell could probably link those up to double down and delete all the items returned with a single command, but I was able to do it manually as I only had a few of these troublesome resources clogging up my migration.

Great Tools: Screen To Gif

Tuesday, May 2, 2017 by Nate Bross

I’m going to use this post to tag a series of great tools that I use. For those times when a screenshot isn’t quite good enough. ScreenToGif is a nice, clean screen capture and gif editor. Its a free download and it just runs. No installer (though it does need .NET 4.6.1) just download, save, and go.

Here’s a screenshot of the tool, around my Live Writer editor:

image

ScreenToGif has multiple capture options, but the one I find most useful is screen. You drag the ScreenToGif window around the screen area you wish to record. You simply hit the record/stop buttons to record while you work, then it opens your project in the editor to make any post-production changes you need. Then you can save as a .gif file for distribution.

Here is a short recording of the above:

ScreenToGifDemo

Often a screenshot is plenty, but sometimes a quick gif communicates so much more.

 

Model Binding with File Uploads using ASP.NET Core Action Method Parameter Attributes

Tuesday, April 11, 2017 by Nate Bross

Started with a simple task: Upload a file to an ASP.NET Core API Controller. The project I’m working on uses a front-end SPA framework, so the file upload is coming from javascript and not directly from an html form post.

First lets look at what I was doing wrong and then we can understand why it was wrong.

A quick peek at the client side code that pushes this data to the controller (note this is actually typescript):

// note photoFiles is bound to via Aurelia binding.
var form = new FormData() for (let i = 0; i < this.photoFiles.length; i++) {
    form.append(`files[${i}]`, this.photoFiles[i]);
}
this.http.fetch(`/api/photoUpload/${this.targetId}`, {
    method: 'post', 
    body: form, 
    headers: new Headers() 
})
.then(response => { console.log(response); });

Since we’re loading our ‘FormData’ object into the ‘body’ of the http post, it made sense to me to wire up the ASP.NET Controller Action as follows:

public async Task photoUpload(
     [FromRoute] Guid propertyId,
     [FromBody] IEnumerable files)
{
     foreach (var file in files)
     {
         var name = file.Name;
         Console.WriteLine(name);
     }

    return new ObjectResult(null);
}

This would result an an HTTP 415 Unsupported Media Type response, before the action code ever executed.

It turns out `[FromBody]` actually kicks ASP.NET Core into JSON Model Binder mode; which obviously cannot handle the file data that is coming through. More information on Model Binding can be found here on Andrew Lock’s blog.

Easy enough; lets drop the `[FromBody]` attribute since its clearly not helping. Without going through the code again, the method is executed; however the files parameter has a count of zero.

At this point, I’ve been searching around enough to have seen a few blog posts that suggest forgoing the Model Binding and simply use the built in

var form = await Request.ReadFormAsync();

var files = form.Files;

method. I try this and it does in fact work. I see my uploaded files. Some folks would stop here with a working solution, but I like to know why something I expected to work didn’t. If the data is coming through, it should work as an action parameter as well as directly reading from the form.

Enter the `[FromForm]` attribute, useful in cases when you want to bind to a specific form field. Modify the above code to use `[FromForm]` on the `IEnumerable<IFormFile>` parameter from our action:

public async Task photoUpload(
    [FromRoute] Guid propertyId,
    [FromForm] IEnumerable files
)

and wait, why isn’t that working? Still getting an empty IEnumerable when the method executes. No Http 415 though, so at least its not a regression.

I went back and compared the ACTUAL http post data that was sent and compared it with a direct form post from a <form method=”post”> and noticed that there was one slight bug in the assembly of the FormData on the front-end:

var form = new FormData()
for (let i = 0; i < this.photoFiles.length; i++) {
     form.append(`files`, this.photoFiles[i]);
}

The original code was doing something like files[0], files[1], etc for each file uploaded, however, the regular <input type=”file”> control simply uses the same ‘name’ as the input tag.

 

 View More Posts