Be careful with search grammar items in string searches

I was caught out the other day using hard-coded string searches in a ServiceAPI application.  It can be convenient (or necessary) to store canned searches (e.g. 'extension:docx OR extension:doc') at times.  The thing to remember is that search strings can be localised and your end user may not be using English.

First, the clauses

Typically the English versions of the search clauses should work irrespective of the user's language, so even if the end user has selected French ' extension' should still be OK.  Due to my over cautious nature I still use the internal name (e.g. 'recExtension) which can be found in the list of search clauses in the ServiceAPI.

Then the grammar items

Search grammar items (such as 'or' and 'and') are not language neutral so 'recExtension:doc or recExtensiom:docx' will not produce any results if your end user has selected Dutch as their language, what you will need is 'recExtension:doc of recExtensiom:docx'.

SDK

Getting the caption for a search grammar item in the SDK is simple:

EnumItem item = new EnumItem(AllEnumerations.SearchGrammarItem, (int)SearchGrammarItem.And).Caption

ServiceAPI - .Net

It is nearly as simple to get the grammar items using the ServiceAPI .Net client.

TrimClient client = new TrimClient("http://localhost/ServiceAPI");
client.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;

var response = client.Get<EnumItemDetailsResponse>(new EnumItemDetails() { Enums = new AllEnumerations[] { AllEnumerations.SearchGrammarItem } });
Console.WriteLine(response.EnumItems[AllEnumerations.SearchGrammarItem].Where(ei => ei.Name == "Not").First().Caption);

ServiceAPI

Or query the ServiceAPI directly:

http://localhost/ServiceAPI/EnumItem?Enums=SearchGrammarItem&format=json

In short

Never hard code string searches using things like 'not', 'or', 'and', 'me' or anything else in the SearchGrammarItems enum.  If you do and someone switches languages then everything is broken.

Using Postman to create a record

Previously I experiment with searching using Postman  Today lets create a record and upload a file in the same request.

Some Code

Here is the HTTP I generated using Postman.

POST /ServiceAPI81/record HTTP/1.1
Host: desktop-39dgcn3
Authorization: Basic aXR1X3RhZG1pbjpUcmltQEhQMQ==
Accept: application/json
Cache-Control: no-cache
Postman-Token: b9575896-9e99-ccf0-4803-452e4fba11d4
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW

------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="RecordRecordType"

Document
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="Files"; filename=""
Content-Type: 


------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="RecordAuthor"

david
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="AlcoholLevel"

5
------WebKitFormBoundary7MA4YWxkTrZu0gW--

ServiceAPI impersonation

The configuration section of the ServiceAPI help documents a handy little property called 'trustedToImpersonate'.  This allows your ServiceAPI to choose to trust one or more calling services.

Scenario

This can be useful in a server to server scenario where passing the actual user credentials to the ServiceAPI is not practical.

Configuration

To allow a particular account to impersonate others users set their name in the hptrim.config file as seen below.  TrustedToImpersonate is a regular expression so you can list multiple accounts if you need to.

<hptrim
  serviceFeatures="Razor,Html,Json,Xml,PredefinedRoutes" 
  trustedToImpersonate="trim\\davidc"
  ...
</hptrim>       

Usage

To impersonate someone using the .Net client librarys use the SetUserToImpersonate() method.

TrimClient client = new TrimClient("http://MyServer/ServiceAPI");
client.Credentials = new NetworkCredential("davidc", "my password", "trim");
client.SetUserToImpersonate("someone\\else");

To impersonate from a different context simply send an HTTP header named 'userToImpersonate' with the value being the user's name along with each request.

Warning when setting the ACL via the SDK

Overview

In the native (or web) client if I attempt to set an invalid ACL on a Record I will get a warning, something like this:

So, I asked myself, why was I not getting the same warning from the SDK?

A riddle

When will the appropriate warning appear in rec.ErrorMessage below...

Record rec = new Record(db, "REC_2");

TrimAccessControlList acl = rec.AccessControlList;
acl.SetAllInherited();
Console.WriteLine("A {0}", rec.ErrorMessage);
rec.AccessControlList = acl;
Console.WriteLine("B {0}", rec.ErrorMessage);
rec.Verify(true);
Console.WriteLine("C {0}", rec.ErrorMessage);
rec.Save();
Console.WriteLine("D {0}", rec.ErrorMessage);

The only point at which a warning message will be present in rec.ErrorMessage is at 'B' above.  Due to the vagaries of the SDK property handling (as described here) calling Verify (or Save() clears out the Error and ErrorMessage properties on the Record object.

The motto of the story

If you are interested in getting the warning from setting the ACL then check for it as soon as you set the AccessControlList property.

ServiceAPI searching, GET or POST

It had been in the back of my mind that using HTTP GET for searching in a project I was working on would lead to trouble and several hours of re-factoring later I am proved correct.

Background

The default way we do searching in the ServiceAPI is using HTTP GET, for example:

http://localhost/ServiceAPI/Record?pageSize=20&q=all&filter=extension:docx,xlsx&properties=RecordTitle&format=json

There are several advantages to use the GET verb but there is also a trap which is that IIS has limitations on the length of a URL.  Usually this is not a problem and the ServiceAPI has design features to help keep URLs concise (such as custom property sets) however I encountered a scenario where I could not avoid the IIS error resulting from an overlong URL.

The scenario

The scenario that caught me was needing the flexibility for the end user to specify as many properties in the properties query parameter as they desired and I did not want to use the 'all' property set as I did not want the cost of fetching every record property.  If the user chose too many properties their request would fail.

The solution

The solution is that the ServiceAPI allows you to POST the same request that you usually GET, you just have to post it to a different end point.  There are a couple of examples of this using an form post and AJAX in the documentation, here is an example using the .Net client.

TrimClient client = new TrimClient("http://localhost/ServiceAPI");
client.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;


PostSearch request = new PostSearch();
request.TrimType = BaseObjectTypes.Record;
request.q = "all";
request.Properties = new PropertyList(PropertyIds.RecordTitle, PropertyIds.RecordNumber);

var response = client.ServiceClient.Send<RecordsResponse>("POST" , "Search", request);

foreach (Record rec in response.Results)
{
    Console.WriteLine(rec.Title);
}

WebDrawer download file from PDF/TIF preview

WebDrawer uses the Trapeze viewer for PDF and TIF files, it was pointed out to me that when in this view the user does not have a menu link to download the original document. What the end users wanted was something like this.

The Solution

I am not sure why we do not have a download link in this context to start with, in the mean-time to add one:

  1. open the file \Views\HTML5_Viewer.cshtml
  2. find the menu code as seen below
  3. add a new LI inside the UL (see 'code to add')

Code to add

<li>
    <a href="~/Record/@record.Uri/File/document">@Translations.lang.document</a>
</li>

IMGF

The big thing I love about getting out to of the office is hearing from our partners and customers about how they use HPE CM.  Meeting face to face helps a backroom person like me feel the passion attached to some of the feature requests we get through the usual channels.  It is not unusual to hear something like 'I have wanted feature X for years but it never seems to make the cut' and then be able to toss that into the mix in our next team meeting and find it is something that fits well into our next release.

So I look forward to catching up in just over a week with anyone who can get to IMGF here in Australia.

Add a count button to WebDrawer

It is possible to speed up the search results in WebDrawer by switching to simpler page navigation.  This post shows how to add a 'Get Count' button to allow users to determine the total number of search results.

Follow the steps in the video and use the code snippets below.

The Code

Views\Shared\searchResults.cshtml

<ul id="trim-pagination">
    <li class="@(start < 2 ? "disabled" : null)">
        @if (start > 1)
            {
                queryString["start"] = (start > pageSize ? start - pageSize : 0).ToString();
            <a href="?@queryString.ToFormUrlEncoded()">&laquo;</a>
        }
        else
        {
            <span> &laquo;</span>
        }
    </li>
    <li>
        @{
            var t = ((start - 1) / pageSize) + 1;
        }
        <span>@t</span>
    </li>

    <li class="@(!this.Model.HasMoreItems ? "disabled" : null)">
        @if (this.Model.HasMoreItems)
            {
                start = (start == 0 ? start + 1 : start);
                queryString["start"] = (start + pageSize).ToString();
            <a href="?@queryString.ToFormUrlEncoded()">&raquo;</a>
        }
        else
        {
            <span>&raquo;</span>
        }
    </li>
    <li>
        <button id="getCount" class="btn search-btn count-btn">Get Count </button>
    </li>
</ul>

css\main.css

.count-btn
{
    margin-left: 20px;
    margin-top:4px;
}

html.busy, html.busy * {  
  cursor: wait !important;  
} 

scripts\webdrawer.js

$("html").bind("ajaxStart", function () {
    $(this).addClass('busy');
}).bind("ajaxStop", function () {
    $(this).removeClass('busy');
});


$("#getCount").click(function () {
    var q = location.href;
    var my_url = location.href + "&format=json&ExcludeCount=true&CountResults=true";
    $.get(my_url, function (response) {
        $("#getCount").text("Count: " + response.Count);
    });
});

WebDrawer Error Message

In WebDrawer 8.2 and 8.3 you might see the error message below...

bad_error.PNG

This error reflects the fact that an error has occurred in the process of parsing an error message, so you have an error but do not know what it is. A better error message might look like this...

To fix this problem edit the hptrim.config file and add the uploadBasePath as seen below.  The uploadBasePath is not used by WebDrawer but the error handling in the underlying framework needs it. 

<hptrim 
    uploadBasePath="[Not Used]"
    ...

Java ServiceAPI client

Prior to Content Manager 9.0 there was no supported Java client for the ServiceAPI, although I did publish an experimental project.  As of 9.0 we are able to use the ServiceStack Java client.  This video demonstrates that client. 

The code

The sample project I created in the video is on our samples site. Beware, in the video I omit the call to 'setAlwaysSendBasicAuthHeaders', you should always call this when using basic authentication.

New in 9.1 - Copy Record Number

Some months ago I jumped on the train to Geelong to attend the TRIM User Group.  It is great to catch up with real live 'Content /Records Manager / TRIM' users.  One simple enhancement that was suggested at this meeting was the ability to copy a record number from the confirm dialog. So here it is in 9.1 patch 1...

Caveat

If a portion of the number requires manual input then only the manual portion will be copied.  I am reliably informed that this will be fixed in 9.1 patch 2.

 

CORS with Basic or Windows Authentication

Update

Along with one of our customers I rethought the approach below to integrate everything into the module, see the sample for more details.

Intro

In the ServiceAPI help there is a sample on how to use the CORS feature.  There are some limitations in this feature that make it unlikely to work for most ServiceAPI users.

Preflight

When using a browser, maybe via jQuery, the odds are that a preflight request will be sent.  This is an HTTP OPTIONS request to confirm that the correct CORS headers are returned.  This request is sent without any authentication.  The problem when using IIS basic or integrated authentication is that this request will be rejected.

The solution

The solution is not to use the CORS feature at all but to fall back on traditional ASP.Net techniques, as detailed in this very helpful post .  

Module

Firstly write a, ASP.Net Module to allow us to get into the pipeline before IIS rejects our request, compile this in Visual Studio and place the DLL in the ServiceAPI bin folder.  All OPTIONS requests will now be returned with the status 200.

public class CORSModule : IHttpModule
{
    public void Dispose()  { }

    public void Init(HttpApplication context)
    {
        context.PreSendRequestHeaders += delegate
        {
            if (context.Request.HttpMethod == "OPTIONS")
            {
                var response = context.Response;
                response.StatusCode = (int)HttpStatusCode.OK;
            }
        };
    }
}

Configure module

Now we wire the module into our service in the system.webServer\modules element...

<modules runAllManagedModulesForAllRequests="true">
  <add name="CORSModule" type="CORSModule" />
</modules>

Set CORS headers

Lastly we add the CORS headers to system.webServer\httpProtocol

<customHeaders>
  <add name="Access-Control-Allow-Methods" value="POST,GET,OPTIONS" />
  <add name="Access-Control-Allow-Origin" value="http://localhost" />
  <add name="Access-Control-Allow-Headers" value="Content-Type,Authorization" />
  <add name="Access-Control-Allow-Credentials" value="true"/>
</customHeaders>

Test it out

Once you have implemented the above the jQuery code like this should be successful in accessing a ServiceAPI instance using basic authentication.

$.ajax({
    beforeSend: function (xhr) {
            xhr.setRequestHeader("Authorization", "Basic " + btoa("USERNAME" + ":" + "PASSWORD"));
    },

    url: "http://davidc2012r2/ServiceAPI82/Record/1?&properties=RecordTitle",
    type: 'GET',
    dataType: 'json',
})
.done(function (response, statusText) {
    
    if (statusText === "success") {
        alert("Record named: " + response.Results[0].RecordTitle.Value + " was found.");
    }
})
.fail(function (xhr) {
    var err = eval("(" + xhr.responseText + ")");
    alert(err.ResponseStatus.Message);

});

WebDrawer - Hide And/Or button in search form

When you create a search form you may choose to only give it one field, in which case the And/Or button is redundant (see below).

The cleanest way to hide this button is to edit the razor template that produces the form.  One simple change is required, first open Views\Search.cshtml (in 8.2 Views\FormSearch.cshtml) and then:

Find the HTML that looks like this:

<label class="radio inline">
    <input type="radio" name="BooleanOperator" value="and" />And
 </label>
<label class="radio inline">
    <input type="radio" name="BooleanOperator" value="or" checked />Or
</label>

And wrap it in an IF statement like this:

@if (searchForm.DefinitionForm.Pages.SelectMany(x => x.PageItems).Count() > 1)
{
    <label class="radio inline">
        <input type="radio" name="BooleanOperator" value="and" />And
    </label>
    <label class="radio inline">
        <input type="radio" name="BooleanOperator" value="or" checked />Or
    </label>
}

Create a new Record from WebDrawer

There are many samples in the ServiceAPI documentation for creating forms to create a new Record, you can adapt any of these to work in WebDrawer if you want to add a record creation form in your WebDrawer instance.  Here is a very simple first form... 

Steps

1. Create file

In the WebDrawer install folder create a new folder, give it any name (I called mine 'My'). Inside that folder create a text file (with a CSHTML extension) with any name (I called mine 'Page.cshtml').

2. Create a Form

Paste the code below into your new file:

@{
    ViewBag.Title = "Create and Upload a Document";

    var recordTypes = this.TrimHelper.Search<RecordType>(
        BaseObjectTypes.RecordType, "usable",
        pageSize: 100, properties:new string[]{"Name"}, sortBy:new string[]{"Name"});
}

<form class="trim-form" method="post" action="~/Record" enctype="multipart/form-data" style="margin-left:20px">
    <input type="hidden" name="Continue" value="Record?q=uri:{0}" />
    <fieldset>
        <legend>Document details</legend>
        <label>Record Type:</label>
        <select name="@PropertyIds.RecordRecordType">
            @foreach (var recordType in recordTypes)
            {
                <option value="@recordType.Uri">@recordType.Name</option>
            }
        </select>
        <label>Title</label>
        <input name="@PropertyIds.RecordTitle" />
        <label>Files:</label>
        <input type="file" name="Files" multiple="multiple" />
        <button type="submit" class="btn">Submit</button>
    </fieldset>
</form>

3. Open the page

Because my folder is called 'My' and my page called 'Page.cshtml' I can open the page like this:

http://MyServer/webdrawer/my/page

Summary

Read through the ServiceAPI documentation and browse the samples for more ideas.  In essence you can create any type of RM object simply by posting a set of correctly named properties to the correct end point.

WebDrawer search form lookup items

As you can see in the image below the lookup items displayed for an Additional Field in a search form are not sorted by name.  This is not a problem when there are only four items but is when there are many.  The good news is that, with a little bit of code, we can fix this.

The process

Find the correct WebDrawer view

In the WebDrawer folder find the file Views\FormSearch.cshtml (or Views\Search.cshtml in 8.3 and later).

Add code to fetch items

Near the top of the page find this line of code:

HttpContext.Current.Response.AppendCookie(new HttpCookie("HPRM_webDrawer_SearchForm", sfName));

On the next line add this code:

    var roadSurfaces = TrimHelper.Search<LookupItem>(
            BaseObjectTypes.LookupItem,
            "lkiSet:\"Road Surfaces\"",
            pageSize:1000,,
            sortBy:new string[] { "lkiName" },
            properties: new string[] { "Name" }).Select(li => li.Name);

Now edit this code so that instead of searching for all the items in the Lookup Set 'Road Surfaces' it uses the name of your lookup set.  If you want to also change the name of the variable from 'roadSurfaces' to something else, if you do this then code you add later will also need to be modified.

 

Create a SELECT element

Find the code that looks like this:

Prepend this code:

if (pageItem.Id == "9000000000")
{
   <select name="@pageItem.ClauseName" multiple class="wd-search-field">
       @foreach (string val in roadSurfaces)
       {
           <option>@val</option>
        }
    </select>
 } else

So that now it looks like this:

Set Field Uri

Modify the code pasted above so that instead of '9000000000' you have the Uri of the additional field you wish to use the lookup items for.

The Result

The result of adding these two code blocks is that you have complete control of the Lookup Items are displayed for a particular field.  You can add the blocks multiple times for multiple fields.

Warning

Keep a record of any changes you make to your WebDrawer templates so that you can re-apply them when you upgrade.

Take care when 'finding' a Record by number

In a recent post I discussed the FindBy method.  This can significantly reduce the load on a server by eliminating unnecessary requests.  When searching for a record by record number, it is faster to find than to search.

Search Examples

This is faster:

/Record/rec_1?format=json

Than this:

/Record?q=number:rec_1&format=json

FindBy Examples

This is faster:

{
    "RecordTitle": "My Title",
    "RecordContainer": {"FindBy":"REC_1"}
}

Than this:

{
    "RecordTitle": "My Title",
    "RecordContainer": {"FindBy":"number:REC_1"}
}

Caveats

Firstly, if you have numeric record numbers they may clash with the record Uri so you will have to do a search and use the 'number' clause.

Omitting the 'number' clause from a FindBy causes the 'default' search to be used which also includes the title, this may result in false positives.

Summary

On systems under heavy load you may consider avoiding the number clause as long as you write your code to filter out any false positive search results that result from doing so.