Sam's Blog » Tutorials

Creating a WPF Tool Window for Visual Studio 2010

280Z28 — Mon, 25 Jan 2010 22:10:18 +0000

I’ve been working on Visual Studio 2010 extensibility for some time now, and I must say that creating a tool window was not the easiest task in the world.

First Attempt: Create an MEF-friendly IToolWindowService

The original goal of this project was creating an MEF service that allowed exporting classes implementing IToolWindowProvider, and have the service manage the creation of the tool windows and their entries on the View > Other Windows menu. This worked, but had several drawbacks that eventually led to the conclusion that this was the wrong approach. First and foremost, this method forced the assemblies providing tool windows to load even when the tool windows weren’t visible. The performance implications of this rule out using MEF as a general solution to this problem. That said, here are some other “little things” that I didn’t have worked out in the MEF solution:

Tool windows would not save their visible state; they were always hidden the next time Visual Studio was opened.
The menu commands to show the tool windows were not available until a source file of {insert some hard-coded type} was opened. This is because the service loaded itself by exporting IVsTextViewCreationListener. When I tried to use the “any” content type, I found that the service would cause an InvalidOperationException when the environment loaded because the tool windows were created while the environment was using an IEnumerator to traverse the current windows.

Overview of the Improved Process

Here is an outline of the general process of creating a new tool window. Following the outline, I’ll explain in detail what each step requires.

Create a VSIX project
Configure the project to offer a VSPackage and a command table
Create the WPF control your tool window will be displaying
Include my helper class WpfToolWindowPane in your project
Derive a class from WpfToolWindowPane for your tool window
Set up menu commands

Create a VSIX project

I’m basing this off of the VSIX project, because it’s a nice clean project template. It’s also beneficial because users that already have a VSIX project can easily add a tool window to it.

Create a project using the “VSIX Project” template (Visual C# > Extensibility). Choose a different solution name than project name or you’ll run into issues later.
Delete the file VSIXProject.cs.
Right click the file “source.extension.manifest” and select View Code. Update the Name, Author, and Description fields with information about your extension. Save and close the file.
Right click the project in Solution Explorer and select Unload Project. Then right click it again and select “Edit MyProject.csproj”
Locate the line that says false and change the value to true. Save and close the file.
Right click the project in Solution Explorer and select Reload Project. If Visual Studio gives an error about a project with the same name already being open, restart Visual Studio and try again.

Configure the project to offer a VSPackage and a command table

Add an XML file to the project named “MyProject.vsct”. In the properties pane for it, change the Build Action to VSCTCompile.
Add a resources file to the project named “VSPackage.resx” (use exactly that name). Open the file in the resources editor, and at the top set the Access Modifier to No code generation. Save and close the file.
Unload the project again and open it for editing.
Set GeneratePkgDefFile to true.
Below the line with GeneratePkgDefFile, add true.
Set CopyBuildOutputToOutputDirectory to true.
Locate the line containing and change it to 1000Designer .
Find the line containing , and add both true and VSPackage.resources as children.
Save, close, and reload the project file.
Add a new class to the project named MyProjectPackage. Derive the class from Microsoft.VisualStudio.Shell.Package. Set the class GUID by adding the attribute [Guid("00000000-0000-0000-0000-000000000000")] (use the Create GUID tool to create the actual value you’ll use here). Add the attribute [PackageRegistration(UseManagedResourcesOnly = true)] to tell Visual Studio to register the class as a VSPackage. Provide the command table (vsct) by adding the attribute [ProvideMenuResource(1000, 1)].

Create the WPF control your tool window will be displaying

For this part, you can create any control derived directly or indirectly from System.Windows.Control.

Include the helper class WpfToolWindowPane in your project

This helper class handles most of the code required for providing a tool window. You can download WpfToolWindowPane.cs here.

Derive a class from WpfToolWindowPane for your tool window

Add a new class named ToolNameToolWindowPane to your project and derive it from WpfToolWindowPane. Use the Create GUID tool to create a guid for ToolNameToolWindowPane, and add a GuidAttribute to it.
In the constructor, set the Caption of your tool window with base.Caption = "Tool Window Name"; .
Override the CreateToolWindowControl() method to return a new instance of the control you created earlier. The method can be as simple as return new ToolNameControl(base.GlobalServiceProvider); .

Set up menu commands

The last step is adding a command to the View > Other Windows menu so your tool window can be opened.

Setting up the command table

Open the MyProject.vsct file and fill it with the following. Use the same Guid for guidMyProjectPackage as you used for the MyProjectPackage class. For the other two placeholders, create new Guids.

< ?xml version="1.0" encoding="utf-8" ?>
 xmlns="http://schemas.microsoft.com/VisualStudio/2005-10-18/CommandTable">
   href="vsshlids.h"/>
   package="guidMyProjectPackage">
    >
       guid="guidMyProjectPackageCmdSet" id="cmdidShowToolName" priority="0x100" type="Button">
        
        >
          >Tool Name>
        >
      >
    >
 
    
  >
 
  >
     guid="guidMyProjectPackageCmdSet" id="cmdidShowToolName" priority="0x100">
       guid="guidSHLMainMenu" id="IDG_VS_WNDO_OTRWNDWS1" />
    >
  >
 
  >
     name="guidMyProjectPackage" value="{00000000-0000-0000-0000-000000000000}" />
 
     name="guidMyProjectPackageCmdSet" value="{00000000-0000-0000-0000-000000000000}">
       name="cmdidShowToolName" value="0x2001" />
    >
 
    
  >
>

Add a Constants class to your project

Add a class named Constants to your project to make the Guids you used in the command table accessible to your code.

internal static class Constants
{
  public const int ToolWindowCommandId = 0x2001;
  public const string MyProjectPackageCmdSet = "{00000000-0000-0000-0000-000000000000}";
  public static readonly Guid GuidMyProjectPackageCmdSet = new Guid(MyProjectPackageCmdSet);
}

Handling the menu command

Open the MyProjectPackage class. Add a new attribute to provide your tool window: [ProvideToolWindow(typeof(ToolNameToolWindowPane))]. Add the following code to the body of the class:

protected override void Initialize()
{
  base.Initialize();
  WpfToolWindowPane.ProvideToolWindowCommand<ToolNameToolWindowPane>(this, Constants.GuidMyProjectPackageCmdSet, Constants.ToolWindowCommandId);
}

MEF support for the Output Window in Visual Studio 2010

280Z28 — Wed, 25 Nov 2009 23:17:29 +0000

As some people have noticed, the Visual Studio 2010 SDK doesn’t have great support for the Output Window from MEF extensions. This post aims to change that. Like several other components I’m writing about, I hope to provide this one in a “general extensibility helper” that could be installed separately from extensions that use it so its features can be shared among them.

There are a couple goals here:

Provide a service for accessing the output window panes
Provide a simple method of creating new output window panes

I’ll start with the usage (since that’s the interface people will normally see) and follow it with the implementation.

Defining an output window pane

Now this is easy.

namespace JavaLanguageService
{
    using System.ComponentModel.Composition;
    using Microsoft.VisualStudio.Utilities;
    using JavaLanguageService.Panes;
 
    public static class Services
    {
        [Export]
        [Name("ANTLR IntelliSense Engine")]
        internal static OutputWindowDefinition AntlrIntellisenseOutputWindowDefinition;
    }
}

Using an output window

First you import the IOutputWindowService:

1 2	[Import] internal IOutputWindowService OutputWindowService;

Then you use it to get an output window, which you can write to:

1
2
3

var outputWindow = OutputWindowService.TryGetPane("ANTLR IntelliSense Engine");
if (outputWindow != null)
    outputWindow.WriteLine(message);

If you want to write to one of the standard panes, pass one of the following to TryGetPane:

public static class PredefinedOutputWindowPanes
{
    public static readonly string General;
    public static readonly string Debug;
    public static readonly string Build;
}

The `IOutputWindowPane` and `IOutputWindowService` interfaces

namespace JavaLanguageService.Panes
{
    using System;
 
    public interface IOutputWindowPane : IDisposable
    {
        string Name
        {
            get;
            set;
        }
 
        void Activate();
        void Hide();
        void Write(string text);
        void WriteLine(string text);
    }
}

namespace JavaLanguageService.Panes
{
    public interface IOutputWindowService
    {
        IOutputWindowPane TryGetPane(string name);
    }
}

namespace JavaLanguageService.Panes
{
    public sealed class OutputWindowDefinition
    {
    }
}

namespace JavaLanguageService.Panes
{
    public static class PredefinedOutputWindowPanes
    {
        public static readonly string General = "General";
        public static readonly string Debug = "Debug";
        public static readonly string Build = "Build";
    }
}

The internal implementation

namespace JavaLanguageService.Panes
{
    internal interface IOutputWindowDefinitionMetadata
    {
        string Name
        {
            get;
        }
    }
}

namespace JavaLanguageService.Panes
{
    using System;
    using System.Collections.Generic;
    using System.ComponentModel.Composition;
    using System.Linq;
    using JavaLanguageService.Extensions;
    using Microsoft.VisualStudio;
    using Microsoft.VisualStudio.Shell.Interop;
    using IOleServiceProvider = Microsoft.VisualStudio.OLE.Interop.IServiceProvider;
 
    [Export(typeof(IOutputWindowService))]
    internal sealed class OutputWindowService : IOutputWindowService
    {
        [Import]
        public IServiceProvider GlobalServiceProvider;
 
        [ImportMany]
        internal List<Lazy<OutputWindowDefinition, IOutputWindowDefinitionMetadata>> OutputWindowDefinitions;
 
        private readonly Dictionary<string, Guid> _outputWindows =
            new Dictionary<string, Guid>()
            {
                { PredefinedOutputWindowPanes.Build, VSConstants.GUID_BuildOutputWindowPane },
                { PredefinedOutputWindowPanes.Debug, VSConstants.GUID_OutWindowDebugPane },
                { PredefinedOutputWindowPanes.General, VSConstants.GUID_OutWindowGeneralPane },
            };
 
        private readonly Dictionary<string, IOutputWindowPane> _panes = new Dictionary<string, IOutputWindowPane>();
 
        public IOutputWindowPane TryGetPane(string name)
        {
            IOutputWindowPane pane = null;
            if (_panes.TryGetValue(name, out pane))
                return pane;
 
            var olesp = (IOleServiceProvider)GlobalServiceProvider.GetService(typeof(IOleServiceProvider));
            var outputWindow = olesp.TryGetGlobalService<SVsOutputWindow, IVsOutputWindow>();
            if (outputWindow == null)
                return null;
 
            Guid guid;
            if (!_outputWindows.TryGetValue(name, out guid))
            {
                var definition = OutputWindowDefinitions.FirstOrDefault(lazy => lazy.Metadata.Name.Equals(name));
                if (definition == null)
                    return null;
 
                guid = Guid.NewGuid();
                // this controls whether the pane is listed in the output panes dropdown list, *not* whether the pane is initially selected
                bool visible = true;
                bool clearWithSolution = false;
 
                if (ErrorHandler.Failed(ErrorHandler.CallWithCOMConvention(() => outputWindow.CreatePane(ref guid, definition.Metadata.Name, Convert.ToInt32(visible), Convert.ToInt32(clearWithSolution)))))
                    return null;
 
                _outputWindows.Add(definition.Metadata.Name, guid);
            }
 
            IVsOutputWindowPane vspane = null;
            if (ErrorHandler.Failed(ErrorHandler.CallWithCOMConvention(() => outputWindow.GetPane(ref guid, out vspane))))
                return null;
 
            pane = new VsOutputWindowPaneAdapter(vspane);
            _panes[name] = pane;
            return pane;
        }
    }
}

namespace JavaLanguageService.Panes
{
    using System;
    using System.Diagnostics.Contracts;
    using Microsoft.VisualStudio;
    using Microsoft.VisualStudio.Shell.Interop;
 
    internal sealed class VsOutputWindowPaneAdapter : IOutputWindowPane, IDisposable
    {
        private IVsOutputWindowPane _pane;
 
        public VsOutputWindowPaneAdapter(IVsOutputWindowPane pane)
        {
            Contract.Requires<ArgumentNullException>(pane != null);
            this._pane = pane;
        }
 
        public string Name
        {
            get
            {
                string name = null;
                ErrorHandler.ThrowOnFailure(this._pane.GetName(ref name));
                return name;
            }
            set
            {
                ErrorHandler.ThrowOnFailure(this._pane.SetName(value));
            }
        }
 
        public void Dispose()
        {
            _pane = null;
        }
 
        public void Activate()
        {
            ErrorHandler.ThrowOnFailure(this._pane.Activate());
        }
 
        public void Hide()
        {
            ErrorHandler.ThrowOnFailure(this._pane.Hide());
        }
 
        public void Write(string text)
        {
            Contract.Requires<ArgumentNullException>(text != null);
            ErrorHandler.ThrowOnFailure(this._pane.OutputStringThreadSafe(text));
        }
 
        public void WriteLine(string text)
        {
            Contract.Requires<ArgumentNullException>(text != null);
            if (!text.EndsWith(Environment.NewLine))
                text += Environment.NewLine;
 
            Write(text);
        }
    }
}

Finally, one extension method that’s used to help with the `IOleServiceProvider`

namespace JavaLanguageService.Extensions
{
    using System;
    using System.Diagnostics.Contracts;
    using System.Runtime.InteropServices;
    using Microsoft.VisualStudio;
    using IOleServiceProvider = Microsoft.VisualStudio.OLE.Interop.IServiceProvider;
 
    public static class ServiceProviderExtensions
    {
        public static TServiceInterface TryGetGlobalService<TServiceClass, TServiceInterface>(this IOleServiceProvider sp)
            where TServiceInterface : class
        {
            Contract.Requires<NullReferenceException>(sp != null);
 
            Guid guidService = typeof(TServiceInterface).GUID;
            Guid riid = typeof(TServiceInterface).GUID;
            IntPtr obj = IntPtr.Zero;
            int result = ErrorHandler.CallWithCOMConvention(() => sp.QueryService(ref guidService, ref riid, out obj));
            if (ErrorHandler.Failed(result) || obj == IntPtr.Zero)
                return null;
 
            try
            {
                TServiceInterface service = (TServiceInterface)Marshal.GetObjectForIUnknown(obj);
                return service;
            }
            finally
            {
                Marshal.Release(obj);
            }
        }
    }
}

A general brace matching tagger for Visual Studio 2010

280Z28 — Wed, 25 Nov 2009 22:32:39 +0000

I haven’t perfected it yet, but here is a fairly general brace matching tagger that seems to work very well. It relies on the classifier for the content type properly tagging comments and literals with classification types derived from PredefinedClassificationTypeNames.Comment and PredefinedClassificationTypeNames.Literal, which any decent classifier will do.

This example really shows a big improvement in Visual Studio 2010. None of my Visual Studio 2008 language services have brace matching this effective and they’re getting jealous.

Here is an example of providing a brace matcher for a “java” content type:

namespace JavaLanguageService
{
    using System.Collections.Generic;
    using System.ComponentModel.Composition;
    using CustomLanguageService.Text;
    using Microsoft.VisualStudio.Text;
    using Microsoft.VisualStudio.Text.Classification;
    using Microsoft.VisualStudio.Text.Editor;
    using Microsoft.VisualStudio.Text.Tagging;
    using Microsoft.VisualStudio.Utilities;
 
    [Export(typeof(IViewTaggerProvider))]
    [ContentType(Constants.JavaContentType)]
    [TagType(typeof(TextMarkerTag))]
    public sealed class JavaBraceMatchingTaggerProvider : IViewTaggerProvider
    {
        [Import]
        public IClassifierAggregatorService AggregatorService;
 
        public ITagger<T> CreateTagger<T>(ITextView textView, ITextBuffer buffer) where T : ITag
        {
            if (textView == null)
                return null;
 
            var aggregator = AggregatorService.GetClassifier(buffer);
            var pairs = new KeyValuePair<char, char>[]
                {
                    new KeyValuePair<char, char>('(', ')'),
                    new KeyValuePair<char, char>('{', '}'),
                    new KeyValuePair<char, char>('[', ']')
                };
            return new BraceMatchingTagger(textView, buffer, aggregator, pairs) as ITagger<T>;
        }
    }
}

And here is the brace matching tagger. Depending on the characteristics of your language, you may be able to use this will little to no modification.

namespace CustomLanguageService.Text
{
    using System;
    using System.Collections.Generic;
    using System.Collections.ObjectModel;
    using System.Diagnostics.Contracts;
    using System.Linq;
    using Microsoft.VisualStudio.Language.StandardClassification;
    using Microsoft.VisualStudio.Text;
    using Microsoft.VisualStudio.Text.Classification;
    using Microsoft.VisualStudio.Text.Editor;
    using Microsoft.VisualStudio.Text.Tagging;
 
    internal sealed class BraceMatchingTagger : ITagger<TextMarkerTag>
    {
        public event EventHandler<SnapshotSpanEventArgs> TagsChanged;
 
        public BraceMatchingTagger(ITextView textView, ITextBuffer sourceBuffer, IClassifier aggregator, IEnumerable<KeyValuePair<char, char>> matchingCharacters)
        {
            Contract.Requires<ArgumentNullException>(textView != null);
            Contract.Requires<ArgumentNullException>(sourceBuffer != null);
            Contract.Requires<ArgumentNullException>(aggregator != null);
            Contract.Requires<ArgumentNullException>(matchingCharacters != null);
 
            this.TextView = textView;
            this.SourceBuffer = sourceBuffer;
            this.Aggregator = aggregator;
            this.MatchingCharacters = matchingCharacters.ToList().AsReadOnly();
 
            this.TextView.Caret.PositionChanged += Caret_PositionChanged;
            this.TextView.LayoutChanged += TextView_LayoutChanged;
        }
 
        public ITextView TextView
        {
            get;
            private set;
        }
 
        public ITextBuffer SourceBuffer
        {
            get;
            private set;
        }
 
        public IClassifier Aggregator
        {
            get;
            private set;
        }
 
        public ReadOnlyCollection<KeyValuePair<char, char>> MatchingCharacters
        {
            get;
            private set;
        }
 
        private SnapshotPoint? CurrentChar
        {
            get;
            set;
        }
 
        private static bool IsInCommentOrLiteral(IClassifier aggregator, SnapshotPoint point, PositionAffinity affinity)
        {
            Contract.Requires(aggregator != null);
 
            // TODO: handle affinity
            SnapshotSpan span = new SnapshotSpan(point, 1);
 
            var classifications = aggregator.GetClassificationSpans(span);
            var relevant = classifications.FirstOrDefault(classificationSpan => classificationSpan.Span.Contains(point));
            if (relevant == null || relevant.ClassificationType == null)
                return false;
 
            return relevant.ClassificationType.IsOfType(PredefinedClassificationTypeNames.Comment)
                || relevant.ClassificationType.IsOfType(PredefinedClassificationTypeNames.Literal);
        }
 
        private bool IsMatchStartCharacter(char c)
        {
            return MatchingCharacters.Any(pair => pair.Key == c);
        }
 
        private bool IsMatchCloseCharacter(char c)
        {
            return MatchingCharacters.Any(pair => pair.Value == c);
        }
 
        private char GetMatchCloseCharacter(char c)
        {
            return MatchingCharacters.First(pair => pair.Key == c).Value;
        }
 
        private char GetMatchOpenCharacter(char c)
        {
            return MatchingCharacters.First(pair => pair.Value == c).Key;
        }
 
        private static bool FindMatchingCloseChar(SnapshotPoint start, IClassifier aggregator, char open, char close, int maxLines, out SnapshotSpan pairSpan)
        {
            pairSpan = new SnapshotSpan(start.Snapshot, 1, 1);
            ITextSnapshotLine line = start.GetContainingLine();
            string lineText = line.GetText();
            int lineNumber = line.LineNumber;
            int offset = start.Position - line.Start.Position + 1;
 
            int stopLineNumber = start.Snapshot.LineCount - 1;
            if (maxLines > 0)
                stopLineNumber = Math.Min(stopLineNumber, lineNumber + maxLines);
 
            int openCount = 0;
            while (true)
            {
                while (offset < line.Length)
                {
                    char currentChar = lineText[offset];
                    // TODO: is this the correct affinity
                    if (currentChar == close && !IsInCommentOrLiteral(aggregator, new SnapshotPoint(start.Snapshot, offset + line.Start.Position), PositionAffinity.Successor))
                    {
                        if (openCount > 0)
                        {
                            openCount--;
                        }
                        else
                        {
                            pairSpan = new SnapshotSpan(start.Snapshot, line.Start + offset, 1);
                            return true;
                        }
                    }
                    // TODO: is this the correct affinity
                    else if (currentChar == open && !IsInCommentOrLiteral(aggregator, new SnapshotPoint(start.Snapshot, offset + line.Start.Position), PositionAffinity.Successor))
                    {
                        openCount++;
                    }
 
                    offset++;
                }
 
                // move on to the next line
                lineNumber++;
                if (lineNumber > stopLineNumber)
                    break;
 
                line = line.Snapshot.GetLineFromLineNumber(lineNumber);
                lineText = line.GetText();
                offset = 0;
            }
 
            return false;
        }
 
        private static bool FindMatchingOpenChar(SnapshotPoint start, IClassifier aggregator, char open, char close, int maxLines, out SnapshotSpan pairSpan)
        {
            pairSpan = new SnapshotSpan(start, start);
            ITextSnapshotLine line = start.GetContainingLine();
            int lineNumber = line.LineNumber;
            int offset = start - line.Start - 1;
 
            // if the offset is negative, move to the previous line
            if (offset < 0)
            {
                lineNumber--;
                line = line.Snapshot.GetLineFromLineNumber(lineNumber);
                offset = line.Length - 1;
            }
 
            string lineText = line.GetText();
 
            int stopLineNumber = 0;
            if (maxLines > 0)
                stopLineNumber = Math.Max(stopLineNumber, lineNumber - maxLines);
 
            int closeCount = 0;
            while (true)
            {
                while (offset >= 0)
                {
                    char currentChar = lineText[offset];
                    // TODO: is this the correct affinity
                    if (currentChar == open && !IsInCommentOrLiteral(aggregator, new SnapshotPoint(start.Snapshot, offset + line.Start.Position), PositionAffinity.Successor))
                    {
                        if (closeCount > 0)
                        {
                            closeCount--;
                        }
                        else
                        {
                            pairSpan = new SnapshotSpan(line.Start + offset, 1);
                            return true;
                        }
                    }
                    // TODO: is this the correct affinity
                    else if (currentChar == close && !IsInCommentOrLiteral(aggregator, new SnapshotPoint(start.Snapshot, offset + line.Start.Position), PositionAffinity.Successor))
                    {
                        closeCount++;
                    }
 
                    offset--;
                }
 
                // move to the previous line
                lineNumber--;
                if (lineNumber < stopLineNumber)
                    break;
 
                line = line.Snapshot.GetLineFromLineNumber(lineNumber);
                lineText = line.GetText();
                offset = line.Length - 1;
            }
 
            return false;
        }
 
        public IEnumerable<ITagSpan<TextMarkerTag>> GetTags(NormalizedSnapshotSpanCollection spans)
        {
            if (spans.Count == 0)
                yield break;
 
            // don't do anything if the current SnapshotPoint is not initialized or at the end of the buffer
            if (!CurrentChar.HasValue || CurrentChar.Value.Position >= CurrentChar.Value.Snapshot.Length)
                yield break;
 
            // hold on to a snapshot of the current character
            var currentChar = CurrentChar.Value;
 
            if (IsInCommentOrLiteral(Aggregator, currentChar, TextView.Caret.Position.Affinity))
                yield break;
 
            // if the requested snapshot isn't the same as the one the brace is on, translate our spans to the expected snapshot
            currentChar = currentChar.TranslateTo(spans[0].Snapshot, PointTrackingMode.Positive);
 
            // get the current char and the previous char
            char currentText = currentChar.GetChar();
            // if current char is 0 (beginning of buffer), don't move it back
            SnapshotPoint lastChar = currentChar == 0 ? currentChar : currentChar - 1;
            char lastText = lastChar.GetChar();
            SnapshotSpan pairSpan = new SnapshotSpan();
 
            if (IsMatchStartCharacter(currentText))
            {
                char closeChar = GetMatchCloseCharacter(currentText);
                /* TODO: Need to improve handling of larger blocks. this won't highlight if the matching brace is more
                 *       than 1 screen's worth of lines away. Changing this to 10 * TextView.TextViewLines.Count seemed
                 *       to improve the situation.
                 */
                if (BraceMatchingTagger.FindMatchingCloseChar(currentChar, Aggregator, currentText, closeChar, TextView.TextViewLines.Count, out pairSpan))
                {
                    yield return new TagSpan<TextMarkerTag>(new SnapshotSpan(currentChar, 1), PredefinedTextMarkerTags.BraceHighlight);
                    yield return new TagSpan<TextMarkerTag>(pairSpan, PredefinedTextMarkerTags.BraceHighlight);
                }
            }
            else if (IsMatchCloseCharacter(lastText))
            {
                var open = GetMatchOpenCharacter(lastText);
                if (BraceMatchingTagger.FindMatchingOpenChar(lastChar, Aggregator, open, lastText, TextView.TextViewLines.Count, out pairSpan))
                {
                    yield return new TagSpan<TextMarkerTag>(new SnapshotSpan(lastChar, 1), PredefinedTextMarkerTags.BraceHighlight);
                    yield return new TagSpan<TextMarkerTag>(pairSpan, PredefinedTextMarkerTags.BraceHighlight);
                }
            }
        }
 
        private void UpdateAtCaretPosition(CaretPosition caretPosition)
        {
            CurrentChar = caretPosition.Point.GetPoint(SourceBuffer, caretPosition.Affinity);
            if (!CurrentChar.HasValue)
                return;
 
            var t = TagsChanged;
            if (t != null)
                t(this, new SnapshotSpanEventArgs(new SnapshotSpan(SourceBuffer.CurrentSnapshot, 0, SourceBuffer.CurrentSnapshot.Length)));
        }
 
        private void TextView_LayoutChanged(object sender, TextViewLayoutChangedEventArgs e)
        {
            if (e.NewSnapshot != e.OldSnapshot)
                UpdateAtCaretPosition(TextView.Caret.Position);
        }
 
        private void Caret_PositionChanged(object sender, CaretPositionChangedEventArgs e)
        {
            UpdateAtCaretPosition(e.NewPosition);
        }
    }
}

Tool tips for squiggles and URLs in Visual Studio 2010

280Z28 — Wed, 25 Nov 2009 21:37:34 +0000

I’ve been experimenting with Visual Studio’s new extensibility model, and ran into a problem with squiggles. It seems that despite having an internal IQuickInfoSource for the SquiggleTag and IUrlTag, it’s not hooked up to a IIntellisenseController so it never appears. Here is a general implementation for an SquiggleQuickInfoIntellisenseController and UrlQuickInfoIntellisenseController that will provide QuickInfo for squiggle tags and URLs provided by custom implementations of ITagger. The controller itself is a generic that supports any ITag that comes with a IQuickInfoSource.

This code is very heavily based on an existing internal IntelliSense controller in Visual Studio 2010 that’s used for a different tag type.

First we have the IIntellisenseControllerProvider for squiggles and URLs:

namespace CustomLanguageServices.Text
{
    using System;
    using System.Collections.Generic;
    using System.ComponentModel.Composition;
    using Microsoft.VisualStudio.Language.Intellisense;
    using Microsoft.VisualStudio.Text;
    using Microsoft.VisualStudio.Text.Editor;
    using Microsoft.VisualStudio.Text.Tagging;
    using Microsoft.VisualStudio.Utilities;
 
    [Export(typeof(IIntellisenseControllerProvider))]
    [ContentType("text")]
    public sealed class SquiggleQuickInfoControllerProvider : IIntellisenseControllerProvider
    {
        [Import]
        public IQuickInfoBroker QuickInfoBroker;
 
        [Import]
        public IViewTagAggregatorFactoryService TagAggregatorFactoryService;
 
        public IIntellisenseController TryCreateIntellisenseController(ITextView textView, IList<ITextBuffer> subjectBuffers)
        {
            Func<TagQuickInfoController<SquiggleTag>> creator =
                () =>
                {
                    var tagAggregator = TagAggregatorFactoryService.CreateTagAggregator<SquiggleTag>(textView);
                    var controller = new TagQuickInfoController<SquiggleTag>(QuickInfoBroker, textView, tagAggregator);
                    return controller;
                };
 
            return textView.Properties.GetOrCreateSingletonProperty(creator);
        }
    }
 
    [Export(typeof(IIntellisenseControllerProvider))]
    [ContentType("text")]
    public sealed class UrlQuickInfoControllerProvider : IIntellisenseControllerProvider
    {
        [Import]
        public IQuickInfoBroker QuickInfoBroker;
 
        [Import]
        public IViewTagAggregatorFactoryService TagAggregatorFactoryService;
 
        public IIntellisenseController TryCreateIntellisenseController(ITextView textView, IList<ITextBuffer> subjectBuffers)
        {
            Func<TagQuickInfoController<IUrlTag>> creator =
                () =>
                {
                    var tagAggregator = TagAggregatorFactoryService.CreateTagAggregator<IUrlTag>(textView);
                    var controller = new TagQuickInfoController<IUrlTag>(QuickInfoBroker, textView, tagAggregator);
                    return controller;
                };
 
            return textView.Properties.GetOrCreateSingletonProperty(creator);
        }
    }
}

And then we have the actual IIntellisenseController:

namespace CustomLanguageService.Text
{
    using Microsoft.VisualStudio.Language.Intellisense;
    using Microsoft.VisualStudio.Text;
    using Microsoft.VisualStudio.Text.Editor;
    using Microsoft.VisualStudio.Text.Tagging;
 
    public class TagQuickInfoController<T> : IIntellisenseController
        where T : ITag
    {
        private IQuickInfoBroker _quickInfoBroker;
        private ITextView _textView;
        private ITagAggregator<T> _tagAggregator;
        private ITextBuffer _surfaceBuffer;
        private IQuickInfoSession _session;
 
        public TagQuickInfoController(IQuickInfoBroker quickInfoBroker, ITextView textView, ITagAggregator<T> tagAggregator)
        {
            this._quickInfoBroker = quickInfoBroker;
            this._textView = textView;
            this._tagAggregator = tagAggregator;
 
            this._surfaceBuffer = textView.TextViewModel.DataBuffer;
            this._surfaceBuffer.Changed += OnSurfaceBuffer_Changed;
            this._textView.MouseHover += OnTextView_MouseHover;
            this._textView.Caret.PositionChanged += OnCaret_PositionChanged;
        }
 
        public void ConnectSubjectBuffer(ITextBuffer subjectBuffer)
        {
        }
 
        public void Detach(ITextView textView)
        {
            this._surfaceBuffer.Changed -= OnSurfaceBuffer_Changed;
            this._textView.MouseHover -= OnTextView_MouseHover;
            this._textView.Caret.PositionChanged -= OnCaret_PositionChanged;
            if (this._tagAggregator != null)
            {
                this._tagAggregator.Dispose();
                this._tagAggregator = null;
            }
        }
 
        public void DisconnectSubjectBuffer(ITextBuffer subjectBuffer)
        {
        }
 
        internal void DismissSession()
        {
            if ((this._session != null) && !this._session.IsDismissed)
            {
                this._session.Dismiss();
                this._session = null;
            }
        }
 
        private bool EnsureSessionStillValid(SnapshotPoint point)
        {
            if (this._session != null)
            {
                if (this._session.IsDismissed)
                {
                    this._session = null;
                    return false;
                }
                if ((this._session.ApplicableToSpan.TextBuffer == point.Snapshot.TextBuffer) && this._session.ApplicableToSpan.GetSpan(point.Snapshot).IntersectsWith(new Span(point.Position, 0)))
                {
                    return true;
                }
                this._session.Dismiss();
                this._session = null;
            }
            return false;
        }
 
        private bool TryExtractQuickInfoFromMarkers(int position)
        {
            IMappingTagSpan<T> mappingTagSpan = null;
 
            foreach (IMappingTagSpan<T> span in this._tagAggregator.GetTags(new SnapshotSpan(this._textView.TextSnapshot, position, 1)))
            {
                if (span.Tag != null)
                    mappingTagSpan = span;
            }
 
            if (mappingTagSpan != null)
            {
                NormalizedSnapshotSpanCollection spans = mappingTagSpan.Span.GetSpans(this._textView.TextBuffer);
                if (spans.Count > 0)
                {
                    this.DismissSession();
                    SnapshotSpan span = spans[0];
                    ITrackingPoint triggerPoint = span.Snapshot.CreateTrackingPoint(span.Start.Position, PointTrackingMode.Positive);
                    this._session = this._quickInfoBroker.CreateQuickInfoSession(this._textView, triggerPoint, true);
                    this._session.Start();
                    return true;
                }
            }
 
            return false;
        }
 
        private void OnSurfaceBuffer_Changed(object sender, TextContentChangedEventArgs e)
        {
            this.DismissSession();
        }
 
        private void OnTextView_MouseHover(object sender, MouseHoverEventArgs e)
        {
            SnapshotPoint? nullable = e.TextPosition.GetPoint(this._surfaceBuffer, PositionAffinity.Successor);
            if (!nullable.HasValue)
                return;
 
            SnapshotPoint point = nullable.Value;
            if (this.EnsureSessionStillValid(point))
                return;
 
            if (this._tagAggregator != null)
            {
                this.TryExtractQuickInfoFromMarkers(e.Position);
            }
        }
 
        private void OnCaret_PositionChanged(object sender, CaretPositionChangedEventArgs e)
        {
            this.DismissSession();
        }
    }
}

Visual Studio language services: Improved brace matching hooks

280Z28 — Wed, 25 Mar 2009 23:29:52 +0000

By default, brace matching is performed in response to the left and right arrow keys. It should be done in response to several additional commands, as shown below.

Movement by clicking the mouse is handled in your override of ViewFilter.HandlePreExec:

public override bool HandlePreExec( ref Guid guidCmdGroup, uint nCmdId, uint nCmdexecopt, IntPtr pvaIn, IntPtr pvaOut ) 
{ 
    if ( guidCmdGroup == VSConstants.VSStd2K ) 
    {
        if ( (VsCommands2K)cmd == VsCommands2K.ECMD_LEFTCLICK )
            base.Source.OnCommand( base.TextView, (VsCommands2K)cmd, '\0' );
    }
 
    return base.HandlePreExec( ref guidCmdGroup, nCmdId, nCmdexecopt, pvaIn, pvaOut );
}

Movement by keystrokes is handled in your override of ViewFilter.HandlePostExec:

public override void HandlePostExec( ref Guid guidCmdGroup, uint nCmdId, uint nCmdexecopt, IntPtr pvaIn, IntPtr pvaOut, bool bufferWasChanged )
{
    if ( guidCmdGroup == VSConstants.VSStd2K )
    {
        switch ( (VsCommands2K)nCmdId )
        {
        case VsCommands2K.LEFT_EXT_COL:
        case VsCommands2K.RIGHT_EXT_COL:
        case VsCommands2K.UP:
        case VsCommands2K.UP_EXT:
        case VsCommands2K.UP_EXT_COL:
        case VsCommands2K.DOWN:
        case VsCommands2K.DOWN_EXT:
        case VsCommands2K.DOWN_EXT_COL:
        case VsCommands2K.HOME:
        case VsCommands2K.HOME_EXT:
        case VsCommands2K.BOL:
        case VsCommands2K.BOL_EXT:
        case VsCommands2K.BOL_EXT_COL:
        case VsCommands2K.FIRSTCHAR:
        case VsCommands2K.FIRSTCHAR_EXT:
        case VsCommands2K.EOL:
        case VsCommands2K.EOL_EXT:
        case VsCommands2K.EOL_EXT_COL:
        case VsCommands2K.LASTCHAR:
        case VsCommands2K.LASTCHAR_EXT:
        case VsCommands2K.PAGEUP:
        case VsCommands2K.PAGEUP_EXT:
        case VsCommands2K.PAGEDN:
        case VsCommands2K.PAGEDN_EXT:
        case VsCommands2K.WORDNEXT:
        case VsCommands2K.WORDNEXT_EXT:
        case VsCommands2K.WORDNEXT_EXT_COL:
        case VsCommands2K.WORDPREV:
        case VsCommands2K.WORDPREV_EXT:
        case VsCommands2K.WORDPREV_EXT_COL:
        case VsCommands2K.BOTTOMLINE:
        case VsCommands2K.BOTTOMLINE_EXT:
        case VsCommands2K.TOPLINE:
        case VsCommands2K.TOPLINE_EXT:
            // check general trigger characters for intellisense
            base.Source.OnCommand( base.TextView, (VsCommands2K)nCmdId, '\0' );
            break;
        }
    }
 
    base.HandlePostExec( ref guidCmdGroup, nCmdId, nCmdexecopt, pvaIn, pvaOut, bufferWasChanged );
}

ManagedMyC: Type and member dropdown bars

280Z28 — Sun, 19 Oct 2008 22:34:10 +0000

This is part 4 of [many?] posts about creating an ANTLR-based language service for Visual Studio.

Now that we have an AST with information about the top-level members in our source files, we can use the tree parser to gather this information and make it available for the dropdown bars. For now, the MyC language doesn’t support user types like structs, enums, or classes, so we’re stuck with listing the members in the file.

Gathering type & member information

The type & member dropdown bars use instances of the DropDownMember class to present their information. For now, we’ll use this class directly in the gathering stage (in the tree parser). The Visual Studio 2008 SDK version 1.0 has a major bug in the DropDownMember class, and using it will crash Visual Studio. You need to use the version 1.1 SDK instead.

Since the language service (MyCLanguageService) is unaware that the parser uses a separate tree parser to process the AST, add the following properties to the MyCParser class (in your helper file for it). We’ll use them to expose the results from the walker.

public IList<DropDownMember> Types
{
    get
    {
        throw new NotImplementedException();
    }
}
 
public IList<DropDownMember> Members
{
    get
    {
        throw new NotImplementedException();
    }
}
 
public MyCWalker Walker
{
    get;
    private set;
}

Since the Types and Members are relevant to a particular source file, we’ll hold these in the MyCSource class, like the braces are. Add the following properties to the MyCSource class:

public IList<DropDownMember> Types
{
    get;
    set;
}
public IList<DropDownMember> Members
{
    get;
    set;
}

The language service moves the types & members to the MyCSource object during the ParseSource request. Find the lines in the MyCLanguageService.ParseSource function where the braces are stored, and store the types & members at the same time:

// store the parse results
source.ParseResult = null;
source.Braces = parser.Braces;
source.Types = parser.Types;
source.Members = parser.Members;

Next, we need to use the tree parser to create a DropDownMember in the Members property for each variable declaration and function definition. To help out, we create the AddType and AddMember functions in MyCWalkerHelper.cs (similar to the Match and Region functions in MyCParserHelper.cs). The helper file becomes:

using System.Collections.Generic;
using Antlr.Runtime;
using Antlr.Runtime.Tree;
using Microsoft.VisualStudio.Package;
using Microsoft.VisualStudio.TextManager.Interop;
 
namespace ManagedMyC
{
    partial class MyCWalker
    {
        public MyCWalker( MyCParser parser, CommonTree tree )
            : this( new CommonTreeNodeStream( tree ) )
        {
            ( (CommonTreeNodeStream)input ).TokenStream = parser.TokenStream;
        }
 
        public List<DropDownMember> UnsortedTypes
        {
            get;
            private set;
        }
        public List<DropDownMember> UnsortedMembers
        {
            get;
            private set;
        }
 
        public void WalkAST()
        {
            UnsortedTypes = new List<DropDownMember>();
            UnsortedMembers = new List<DropDownMember>();
            program();
        }
 
        void AddType( string name, IToken start, IToken stop, IconImageIndex glyph )
        {
            if ( name == null || start == null || stop == null )
                return;
 
            DropDownMember type = new DropDownMember(
                name,
                TextSpanHelper.Merge( MyCParser.ToTextSpan( start ), MyCParser.ToTextSpan( stop ) ),
                (int)glyph,
                DROPDOWNFONTATTR.FONTATTR_PLAIN );
 
            UnsortedTypes.Add( type );
        }
        void AddMember( string name, IToken start, IToken stop, IconImageIndex glyph )
        {
            if ( name == null || start == null || stop == null )
                return;
 
            DropDownMember member = new DropDownMember(
                name,
                TextSpanHelper.Merge( MyCParser.ToTextSpan( start ), MyCParser.ToTextSpan( stop ) ),
                (int)glyph,
                DROPDOWNFONTATTR.FONTATTR_PLAIN );
 
            UnsortedMembers.Add( member );
        }
    }
}

We update the tree parser to use these functions:

tree grammar MyCWalker;

options
{
    language=CSharp2;
    tokenVocab=MyC;
    ASTLabelType=CommonTree;
}

@namespace { ManagedMyC }

program
    :   declarations
    ;

declarations
    :   declaration*
    ;

declaration
    :   declaration_
    ;

declaration_
    :   ^(  AST_FUNCDEF
            class1? type?
            IDENTIFIER
            p=parameters
            block
          {
                AddMember(
                    string.Format( "{0}({1})", $IDENTIFIER.text, $p.ptext ),
                    $IDENTIFIER.Token,
                    $block.close,
                    IconImageIndex.Method
                    );
            }
        )
    |   simple_declaration
    ;

simple_declarations1
    :   simple_declaration+
    ;

simple_declaration
    :   ^(  AST_DECL class1? type
            (  IDENTIFIER
                { AddMember($IDENTIFIER.text, $IDENTIFIER.Token, $IDENTIFIER.Token, IconImageIndex.Variable); }
            )+
        )
    ;

parameters returns [string ptext]
@init
{
    System.Collections.Generic.List param_list = new System.Collections.Generic.List();
}
    :   ^(  AST_PARAMS
            (  parameter
                { param_list.Add($parameter.text); }
            )*
        )
        { $ptext = string.Join( ", ", param_list.ToArray() ); }
    ;

parameter
    :   ^(type IDENTIFIER)
    ;

class1
    :   'static'
    |   'auto'
    |   'extern'
    ;

type
    :   'int'
    |   'void'
    ;

block returns [IToken close]
    :   open_block /*block_content1?*/ close_block
        { $close = $close_block.start.Token; }
    ;

open_block
    :   '{'
    ;

close_block
    :   '}'
    ;

The final step is to implement the Types & Members properties in MyCParserHelper.cs to return the information from the walker.

public IList<DropDownMember> Types
{
    get
    {
        if ( Walker == null || Walker.UnsortedTypes == null )
            return new DropDownMember[0];
        return Walker.UnsortedTypes.OrderBy( type => PositionIndicator( type ) ).ToArray();
    }
}
 
public IList<DropDownMember> Members
{
    get
    {
        if ( Walker == null || Walker.UnsortedMembers == null )
            return new DropDownMember[0];
        return Walker.UnsortedMembers.OrderBy( member => PositionIndicator( member ) ).ToArray();
    }
}
 
static int PositionIndicator( DropDownMember member )
{
    return ( member.Span.iStartLine << 10 ) + member.Span.iStartIndex;
}

Using the type & member information for type & member dropdown bars

The TypeAndMemberDropdownBars class is abstract, so we need to derive our own class from it.

using Microsoft.VisualStudio.Package;
using Microsoft.VisualStudio.TextManager.Interop;
 
using ArrayList = System.Collections.ArrayList;
 
namespace ManagedMyC
{
    public class MyCTypeAndMemberDropdownBars : TypeAndMemberDropdownBars
    {
        public MyCTypeAndMemberDropdownBars( MyCLanguageService languageService )
            : base( languageService )
        {
        }
 
        public override bool OnSynchronizeDropdowns( LanguageService languageService, IVsTextView textView, int line, int col, ArrayList dropDownTypes, ArrayList dropDownMembers, ref int selectedTypeIndex, ref int selectedMemberIndex )
        {
            // TODO
            return false;
        }
    }
}

To use this new class, we overload the CreateDropDownHelper function in MyCLanguageService.

public override TypeAndMemberDropdownBars CreateDropDownHelper( IVsTextView forView )
{
    return new MyCTypeAndMemberDropdownBars( this );
}

We’ll also need some helper functions in the MyCSource class to filter the available types and members information. This information may seem like overkill at this point, but as I mentioned earlier, my intention with this post is to demonstrate a “solid implementation of the TypeAndMemberDropdownBars“. This method has proven powerful, flexible, and reliable for several languages so far.

#region Type & member dropdowns support
/// 
/// Gets the members of the current Source to show in the Types list of the
/// TypeAndMemberDropdownBars.
/// 
/// An enumeration of items to show in the Types dropdown.
public virtual IEnumerable<DropDownMember> GetDropDownTypes()
{
    if ( Types == null )
        return null;
 
    return ( from type in Types
             orderby type.Label
             select type )
           .ToArray();
}
/// 
/// Gets the members of the selected type to show in the Members list of the
/// TypeAndMemberDropdownBars.
/// 
/// The item currently selected in the Types dropdown.
/// A list of TextSpans that are used to filter the list of members.
/// Typically, these spans are sub-spans of the selected type representing nested types, where
/// members within those spans are actually members of a nested type ant should not be shown.
/// An enumeration of items to show in the Members dropdown.
public virtual IEnumerable<DropDownMember> GetDropDownMembers( DropDownMember type, IEnumerable<TextSpan> excludeSpans )
{
    if ( Members == null )
        return null;
 
    var members = ( from member in Members.ToArray()
                    where (type == null) || TextSpanHelper.IsEmbedded( member.Span, type.Span )
                    where !excludeSpans.Any( span => TextSpanHelper.IsEmbedded( member.Span, span ) )
                    orderby member.Label
                    select member )
                  .ToArray();
 
    return members;
}
 
/// 
/// Gets the default item to show in the Types dropdown of the TypeAndMemberDropdownBars.
/// This function is called when the cursor is not in the span of any type in the list.
/// 
/// The types shown in the Types dropdown.
/// The index of the type that should be visible in the Types dropdown.
public virtual int GetDefaultType( IEnumerable<DropDownMember> types )
{
    return ( types.FirstOrDefault() == null ) ? -1 : 0;
}
/// 
/// Gets the default item to show in the Members dropdown of the TypeAndMemberDropdownBars.
/// This function is called when the cursor is not inside the span of any members of the
/// selected type.
/// 
/// The item currently selected in the Types dropdown.
/// The members of the selected type shown in the Members dropdown.
/// The index of the member that should be visible in the Members dropdown.
public virtual int GetDefaultMember( DropDownMember selectedType, IEnumerable<DropDownMember> members )
{
    return ( members.FirstOrDefault() == null ) ? -1 : 0;
}
#endregion

And now for the beast: the implementation of OnSynchronizeDropdowns. Side note: if you find this code helpful, it would make me feel good if you drop me a thanks - there's a lot of work in this.

using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.Package;
using Microsoft.VisualStudio.TextManager.Interop;
 
using ArrayList = System.Collections.ArrayList;
 
namespace ManagedMyC
{
 
    public class MyCTypeAndMemberDropdownBars : TypeAndMemberDropdownBars
    {
        public MyCTypeAndMemberDropdownBars( MyCLanguageService languageService )
            : base( languageService )
        {
        }
 
        // If you find this code helpful, it would make me feel good if you drop me a thanks - there's a lot of work in this.
        public override bool OnSynchronizeDropdowns( LanguageService languageService, IVsTextView textView, int line, int col, ArrayList dropDownTypes, ArrayList dropDownMembers, ref int selectedTypeIndex, ref int selectedMemberIndex )
        {
            MyCSource source = languageService.GetSource( textView ) as MyCSource;
            if ( source == null )
                return false;
 
            DropDownMember selectedType = null;
            DropDownMember selectedMember = null;
 
            if ( selectedTypeIndex >= 0 && dropDownTypes.Count > selectedTypeIndex )
                selectedType = dropDownTypes[selectedTypeIndex] as DropDownMember;
 
            if ( selectedMemberIndex >= 0 && dropDownMembers.Count > selectedMemberIndex )
                selectedMember = dropDownMembers[selectedMemberIndex] as DropDownMember;
 
            IEnumerable<DropDownMember> types = source.GetDropDownTypes();
            if ( types == null )
                return false;
 
            if ( source.TypesUpdated )
            {
                dropDownTypes.Clear();
                dropDownTypes.AddRange( types.ToList() );
                source.TypesUpdated = false;
            }
 
            DropDownMember typeScope = ( from type in types
                                         where TextSpanHelper.ContainsInclusive( type.Span, line, col )
                                         orderby ( type.Span.iStartLine << 16 ) + type.Span.iStartIndex
                                         select type )
                                       .LastOrDefault();
 
            if ( typeScope != null )
                selectedTypeIndex = dropDownTypes.IndexOf( typeScope );
            else if ( selectedTypeIndex < 0 || selectedTypeIndex >= dropDownTypes.Count )
                selectedTypeIndex = source.GetDefaultType( types );
 
            IEnumerable<TextSpan> excludeSpans = null;
            if ( typeScope != null )
            {
                // exclude sub-spans so members of nested types don't show as members of the current type
                excludeSpans = from type in types
                               where TextSpanHelper.IsEmbedded( type.Span, typeScope.Span )
                               where !TextSpanHelper.IsSameSpan( typeScope.Span, type.Span )
                               select type.Span;
            }
            else
            {
                excludeSpans = Enumerable.Empty<TextSpan>();
            }
 
            IEnumerable<DropDownMember> members = source.GetDropDownMembers( typeScope, excludeSpans );
            if ( members == null )
                return false;
 
            if ( source.MembersUpdated || selectedType != typeScope )
            {
                dropDownMembers.Clear();
                dropDownMembers.AddRange( members.ToList() );
                source.MembersUpdated = false;
            }
 
            DropDownMember memberScope = ( from member in members
                                           where TextSpanHelper.ContainsInclusive( member.Span, line, col )
                                           orderby ( member.Span.iStartLine << 16 ) + member.Span.iStartIndex
                                           select member )
                                         .LastOrDefault();
 
            if ( memberScope != null )
                selectedMemberIndex = dropDownMembers.IndexOf( memberScope );
            else if ( selectedMemberIndex < 0 || selectedMemberIndex >= dropDownMembers.Count )
                selectedMemberIndex = source.GetDefaultMember( selectedType, members );
 
            if ( selectedTypeIndex >= 0 )
            {
                // gray out the displayed type if the cursor is no longer in its span
                ( (DropDownMember)dropDownTypes[selectedTypeIndex] ).FontAttr =
                    ( typeScope == null ) ? DROPDOWNFONTATTR.FONTATTR_GRAY : DROPDOWNFONTATTR.FONTATTR_PLAIN;
            }
            if ( selectedMemberIndex >= 0 )
            {
                // gray out the displayed member if the cursor is no longer in its span
                ( (DropDownMember)dropDownMembers[selectedMemberIndex] ).FontAttr =
                    ( memberScope == null ) ? DROPDOWNFONTATTR.FONTATTR_GRAY : DROPDOWNFONTATTR.FONTATTR_PLAIN;
            }
 
            return true;
        }
    }
}

Keeping the type & member dropdown bars in sync with the cursor

The default LanguageService implementation calls SynchronizeDropdowns sometimes, but not every time it's needed. To help with the overall feel of the dropdowns, we need to add two more pieces.

First, we need to call SynchronizeDropdowns after a parse is completed. We already have a property UpdateDropdowns inside the MyCLanguageService class that is checked inside OnIdle. We just need to add the following code in ParseSource, inside the switch ( req.Reason ) under case ParseReason.Check:

1 2	// make sure the dropdowns are updated after a parse UpdateDropdowns = true;

Second, we need to keep track of when our lists of DropDownMembers in the MyCSource class change:

IList<DropDownMember> _types;
IList<DropDownMember> _members;
 
public bool TypesUpdated
{
    get;
    set;
}
public bool MembersUpdated
{
    get;
    set;
}
public IList<DropDownMember> Types
{
    get
    {
        return _types;
    }
    set
    {
        _types = value;
        TypesUpdated = true;
    }
}
public IList<DropDownMember> Members
{
    get
    {
        return _members;
    }
    set
    {
        _members = value;
        MembersUpdated = true;
    }
}

Source code for this article

Here's the source code for the ManagedMyC sample at this point. Since I surely missed things, you can always diff this code versus the original source from my first post on this subject.

Once again, you’ll have to generate your own LanguageService and Package Guids and set them in MyCConstants.cs before building this project.

The file is compressed with 7-zip because it’s free & awesome.
ManagedMyC-2.7z

ManagedMyC: Intro to building an AST

280Z28 — Sat, 18 Oct 2008 21:23:53 +0000

This is part 3 of [many?] posts about creating an ANTLR-based language service for Visual Studio.

Sure a scanner and parser are cool, and syntax highlighting is nice. But the real power in the Visual Studio language services comes in their IntelliSense abilities, and supporting those effectively requires building and processing an AST. In this section, I’ll show how to integrate the ANTLR automatic AST features, including a tree parser, into the ManagedMyC language service. Among other things, I’ll assume the reader is already familiar with ANTLR syntax for parsers, rewrites, and tree parsers.

There are plenty of other places to learn about those separately; my goal is to show how to start incorporating the existing knowledge into a usable language service. This article only discusses adding the tree grammar to your language service and using it to process a bare-bones tree created in the parser. For this article, there are no new UI/language service features supported. In the next article, I’ll show how to use the tree parser output implement a solid TypeAndMemberDropdownBars implementation.

Building an AST in the parser

At this point, we’ll focus on a minimal AST needed to support a limited number of features. We want to know:

The type and name of declared global variables
The signature and location (span) of functions defined at global scope

In the grammar MyC.g3, add options indicating we intend to build an AST.

options
{
    language=CSharp2;
    output=AST;
    ASTLabelType=CommonTree;
}

Then build the minimal AST. Pseudo-tokens are used as the root of some trees for both identification and to keep the tree parser LL(1) for speed. Only the parser rules are included here, and I left out most of the tokens to save space.

tokens
{
    // all our previous tokens still go here
    END_BLOCK_COMMENT = '*/';

    AST_FUNCDEF;
    AST_PARAMS;
    AST_DECL;
}

@lexer::namespace { ManagedMyC }
@parser::namespace { ManagedMyC }

program
    :   declarations
        EOF
    ;

declarations
    :   declaration*
    ;

declaration
    :   declaration_
    ;

declaration_
    :   class1? type? IDENTIFIER paren_params block
        { Region( $block.start, $block.stop ); }
        -> ^(AST_FUNCDEF class1? type? IDENTIFIER paren_params block)
    |   simple_declaration
    ;

simple_declarations1
    :   simple_declaration+
    ;

simple_declaration
    :   semi_declaration ';'!
    ;

semi_declaration
    :   class1? type IDENTIFIER
        (',' IDENTIFIER)*
        -> ^(AST_DECL class1? type IDENTIFIER+)
    ;

params1
    :   parameter (','! parameter)*
    ;

parameter
    :   type^ IDENTIFIER
    ;

paren_params
    :   l='(' params1? r=')'
        { Match($l, $r); }
        -> ^(AST_PARAMS params1?)
    ;

class1
    :   'static'
    |   'auto'
    |   'extern'
    ;

type
    :   'int'
    |   'void'
    ;

// for now, we don't need to know the contents of a block, just where it starts and ends.
block
    :   open_block (block_content1!)? close_block
        { Match($open_block.start, $close_block.start); }
    ;

open_block
    :   '{'
    ;

close_block
    :   '}'
    ;

// From here on it's the same as in the first version of the grammar

Adding a tree grammar to the project

We’ll name the tree grammar MyCWalker, so the grammar file is MyCWalker.g3, the generated class is MyCWalker.cs, and the user-created partial class we’ll call MyCWalkerHelper.cs. When I add a new grammar to a project, the first thing I do is create empty files with those names and place them in the project folder. Then I unload the project in Visual Studio, and right click > Edit ManagedMyC.csproj. Add the following items to the project:

  >
     Include="MyCWalker.g3">
      >MyCWalker.cs;MyCWalker.tokens>
    >
     Include="MyCWalker.cs">
      >True>
      >True>
      >MyCWalker.g3>
    >
     Include="MyCWalkerHelper.cs">
      >MyCWalker.g3>
    >
  >

Save the modified project file, close it, and then right click on the project in Solution Explorer > Reload.

Creating a tree grammar to match the parser’s generated AST

tree grammar MyCWalker;

options
{
    language=CSharp2;
    tokenVocab=MyC;
    ASTLabelType=CommonTree;
}

@namespace { ManagedMyC }

program
    :   declarations
    ;

declarations
    :   declaration*
    ;

declaration
    :   declaration_
    ;

declaration_
    :   ^(AST_FUNCDEF class1? type? IDENTIFIER parameters block)
    |   simple_declaration
    ;

simple_declarations1
    :   simple_declaration+
    ;

simple_declaration
    :   ^(AST_DECL class1? type IDENTIFIER+)
    ;

parameters
    :   ^(AST_PARAMS parameter*)
    ;

parameter
    :   ^(type IDENTIFIER)
    ;

class1
    :   'static'
    |   'auto'
    |   'extern'
    ;

type
    :   'int'
    |   'void'
    ;

// notice we excluded the block_content1 tree in the parser, so we skip it here too
block
    :   open_block /*block_content1?*/ close_block
    ;

open_block
    :   '{'
    ;

close_block
    :   '}'
    ;

Calling the tree parser during parse operations

First, we edit the MyCWalkerHelper.cs file to add a WalkAST function that the parser can use.

using Antlr.Runtime.Tree;
 
namespace ManagedMyC
{
    partial class MyCWalker
    {
        public MyCWalker( MyCParser parser, CommonTree tree )
            : this( new CommonTreeNodeStream( tree ) )
        {
            ( (CommonTreeNodeStream)input ).TokenStream = parser.TokenStream;
        }
 
        public void WalkAST()
        {
            program();
        }
    }
}

The main entry point to the parser is MyCParser.Parse() in MyCParserHelper.cs. We add code to that function to automatically process the tree after a parse:

public bool Parse()
{
    program_return result = program();

    MyCWalker walker = new MyCWalker( this, (CommonTree)result.Tree );
    walker.WalkAST();

    return true;
}

What’s next

I’m not including a new set of source code with this post because the next set of source already includes the TypeAndMemberDropdownBars implementation that I’ll be explaining in the next article.

ManagedMyC: Code folding for functions

280Z28 — Fri, 17 Oct 2008 06:03:41 +0000

This is part 2 of [many?] posts about creating an ANTLR-based language service for Visual Studio.

Since this is a near-trivial feature to add based on the code in my previous post, I’ll go ahead and get it in tonight. We want to fold from the opening { to the closing } of a function, so the first thing to do is find that in the grammar.

declaration_
    :   class1? type? IDENTIFIER paren_params block
    |   simple_declaration
    ;

The first alternative in the declaration_ rule is a function: clearly you have return type, name, parameters, and then the definition block. Since the block rule starts with { and ends with }, it’s exactly what we want to fold. We’ll make this work just like the braces; change the rule to add an action to mark the folding region:

declaration_
    :   class1? type? IDENTIFIER paren_params block
        { Region( $block.start, $block.stop ); }
    |   simple_declaration
    ;

Then open up your MyCParserHelper.cs file and add these two Region helper functions immediately above the DefineRegion function already there:

public void Region( IToken lh, IToken rh )
{
    DefineRegion( TextSpanHelper.Merge( ToTextSpan( lh ), ToTextSpan( rh ) ) );
}
public void Region( IToken token )
{
    DefineRegion( ToTextSpan( token ) );
}

Finally, you need to make sure the AuthoringSink knows that you are making changes to the document’s hidden region markers by adding a line to DefineRegion:

public void DefineRegion( TextSpan span )
{
    Sink.ProcessHiddenRegions = true;
    Sink.AddHiddenRegion( span );
}

Custom Visual Studio language services: ManagedMyC meets ANTLR

280Z28 — Fri, 17 Oct 2008 03:25:35 +0000

As some of you know (ok probably not many of you), I’m the author behind Pixel Mine nFringe, a custom language service framework that we used to provide UnrealScript editing & debugging features in Visual Studio 2005 and 2008. To date, I’ve written 2 full language services with it (UnrealScript and Antlr v3) and toyed with several others (INI files, C/C++, StringTemplate, and a scripting language used in another game). Several people have asked how to get started on a language service using ANTLR grammars for the backend features.

Just to get started, I’ve made a near direct port of the ManagedMyC sample from the Visual Studio SDK, which uses MPLEX/MPPG, to one that uses ANTLR. The most important thing to note at this point: many parts of this sample are inefficient, clumsy, and/or just done the wrong way. None of the features from earlier posts here are implemented in this sample [yet]. Over the next several weeks, I plan to make blog entries covering individual tasks required to make ManagedMyC a solid example of how someone could make a custom language service.

The source code for this post is linked at the end of this article. I’ve divided this post into the two major items involved in creating an ANTLR-based language service: setting up the grammar compiler with MSBuild, and creating a scanner-friendly lexer.

Setting up ANTLR grammars to compile with your language service

Antlr generates .cs files that must be included in the project build. Visual Studio’s build system has a problem identifying files generated in the middle of a build, so you normally have to build your project twice for a grammar change to take effect. To stop Visual Studio from caching the timestamps of the generated files so the build always works, add the following before the Import elements near the end of the project file:

1
2
3

>
  >False>
>

You also need to add rules to have the grammars build with MSBuild. Add the following immediately after the Import elements:

>
  >$(MSBuildProjectDirectory)\..\Antlr>
  >$(CoreCompileDependsOn);GenerateAntlrCode>
  >$(CoreCleanDependsOn);CleanAntlrCode>
>
 Name="GenerateAntlrCode" Inputs="@(Antlr3)" Outputs="%(OutputFiles)">
   Importance="normal" Text="Antlr: Transforming '@(Antlr3)' to '%(Antlr3.OutputFiles)'" />
   Command="java -cp %22$(Antlr3ToolPath)\antlr3.jar;$(Antlr3ToolPath)\antlr-2.7.7.jar;$(Antlr3ToolPath)\stringtemplate-3.1b1.jar%22 org.antlr.Tool -lib %22%(RootDir)%(Directory).%22 -message-format vs2005 @(Antlr3)" Outputs="%(OutputFiles)" />
>
 Name="CleanAntlrCode">
  >
    <_CleanAntlrFileWrites Include="@(Antlr3->'%(RelativeDir)%(Filename).tokens')" />
  >
   Importance="normal" Text="Antlr: Deleting output files '@(_CleanAntlrFileWrites)'" />
  
  
>

`IScanner`-friendly lexers

The biggest difference between traditional parsing and the code highlighter in Visual Studio is in the amount of text they process at a time. For regular parsing, generally started by a call to LanguageService.ParseSource, you have access to and can parse the entire source file as a single block of text. The syntax highlighter, exposed by a class that implements the IScanner interface, only has access to one line at a time, so it must be resumable (better word?) from any point in the file. ANTLR generated lexers are not trivial to use one line at a time, but with a bit of thought up front, it’s not bad either.

The primary problem lies in single lexical tokens, such as block comments, that span more than one line. To work in a Visual Studio colorizer, your lexer needs to be able to start in the middle of any multi-line token. ManagedMyC has 2 tokens that can span multiple lines: white space and block comments.

White space, the not-so-multiline multiline token

First, handling white space. If you have a whitespace rule like the following, simply break it in two so it doesn’t span multiple lines.

Before:

WS
    :   (' '|'\t'|'\r'|'\n')+
        { $channel=HIDDEN; }
    ;

After:

NEWLINE
    :   '\r'? '\n'
        { $channel = HIDDEN; }
    ;

WS
    :   (' '|'\t')+
        { $channel=HIDDEN; }
    ;

Multi-line tokens: block comments

Edit 4/15/09: I have a new method for handling this type of token that is more reliable, faster, and cleaner than this method. I’ll be making a new blog post soon to revisit the issue.

Next is the big one: handling C-style block comments. The following is the simplest form of C-style comment represented in an ANTLR lexer rule:

COMMENT
    :   '/*' .* '*/'
        { $channel=HIDDEN; }
    ;

First, you don’t want your lexer to throw exceptions when a block comment doesn’t end before the end of a line, so we modify the rule to continue to the end of the text or until the comment ends, whichever comes first:

COMMENT
    :   '/*'
        (   ~('*')
            |   ('*' ~'/') => '*'
        )*
        ('*/')?
        { $channel=HIDDEN; }
    ;

Making the lexer able to resume parsing in the middle of a comment is a bit more difficult. We do this by forcing everything to be a comment until we reach the end of the block comment, and then continue like normal. This takes several steps.

First, we need to be able to detect the end of a block comment. Since */ is only valid in C as the end of a block comment, we simply add it in the tokens{} section of the grammar:

tokens
{
    // ...
    END_BLOCK_COMMENT = '*/';
    // ...
}

Next, we need to make sure no character bombs the lexer (NoViableAltException), so we add the following as the last rule in the file:

ANYCHAR
    :   .
    ;

Next, we need to have a property in the lexer class to get/set the current state. In the MyCLexerHelper.cs file (the user-edited file for the partial class), we add the following:

using Antlr.Runtime;
 
namespace ManagedMyC
{
    partial class MyCLexer
    {
        public bool InBlockComment
        {
            get;
            set;
        }
    }
}

which we manipulate inside the COMMENT rule:

COMMENT
    :   '/*'
        {
            InBlockComment = true;
            $channel = HIDDEN;
        }
        (   ~('*')
            |   ('*' ~'/') => '*'
        )*
        ('*/' {InBlockComment = false;})?
    ;

The final step is to override the NextToken() function to handle cases where the lexer started in a comment state:

using Antlr.Runtime;
 
namespace ManagedMyC
{
    partial class MyCLexer
    {
        public bool InBlockComment
        {
            get;
            set;
        }
 
        public override IToken NextToken()
        {
            IToken next = base.NextToken();
 
            if ( next.Type != EOF && InBlockComment && next.Type != COMMENT )
            {
                if ( next.Type == END_BLOCK_COMMENT )
                    InBlockComment = false;
 
                next.Type = COMMENT;
                next.Channel = HIDDEN;
            }
 
            return next;
        }
    }
}

Source code for this sample

You’ll have to generate your own LanguageService and Package Guids and set them in MyCConstants.cs before building this project.

The file is compressed with 7-zip because it’s awesome.
ManagedMyC-1.7z

Custom Visual Studio language services: Tracking recently used items in autocompletion lists

280Z28 — Tue, 07 Oct 2008 20:34:04 +0000

The C# language service has the great feature of remembering recently used items in the completion lists (auto-complete, complete word, member select, etc.). You can add a similar ability to your language service by deriving your Declarations-derived class from MruDeclarations instead of Declarations.

public abstract class MruDeclarations : Declarations
{
    static List<string> _recentCompletions = new List<string>();
    static void InsertUsedItem( string name )
    {
        if ( string.IsNullOrEmpty( name ) )
            return;
 
        if ( _recentCompletions.Count >= 20 )
            _recentCompletions.RemoveAt( 0 );
        _recentCompletions.Remove( name );
        _recentCompletions.Add( name );
    }
 
    public override void GetBestMatch( string value, out int index, out bool uniqueMatch )
    {
        base.GetBestMatch( value, out index, out uniqueMatch );
 
        if ( !uniqueMatch )
        {
            List<string> relevantRecentItems = _recentCompletions.Where( item => item.StartsWith( value, StringComparison.OrdinalIgnoreCase ) ).ToList();
            if ( relevantRecentItems.Count == 0 )
                return;
 
            string last = relevantRecentItems.OrderBy( i => i, StringComparer.OrdinalIgnoreCase ).Last();
 
            var declaration =
                // start at index to ignore items that fall before the first item that starts with the text typed so-far (case-insensitive)
                Enumerable.Range( index, GetCount()-1 )
                // anonymous type to cache the DisplayText
                .Select( i => new
                {
                    DisplayText = GetDisplayText( i ),
                    Index = i
                } )
                // stop taking items when we reach the last relevant recent completion in alphabetical order (case-insensitive)
                .TakeWhile( item => string.Compare( item.DisplayText, last, StringComparison.OrdinalIgnoreCase ) <= 0 )
                // filter on whether the recent completions list contains the item (case-insensitive)
                .Where( item => relevantRecentItems.Contains( item.DisplayText, StringComparer.OrdinalIgnoreCase ) )
                // rank the items that match a recent completion
                //  - give preference to items that were more recently typed
                //  - give preference to case-sensitive matches
                //  => higher rank values indicate better matches
                .OrderByDescending( item =>
                {
                    int withCase = relevantRecentItems.LastIndexOf( item.DisplayText );
                    // bias case-insensitive matches to case-sensitive matches have higher rank
                    int withoutCase = relevantRecentItems.FindLastIndex( i => i.Equals( item.DisplayText, StringComparison.OrdinalIgnoreCase ) ) - 1;
                    // Give preference to the more recent match. Note that LastIndexOf() returns -1 if the item wasn't found,
                    // so if the item doesn't have a case-sensitive match, withoutCase always has the higher value.
                    return Math.Max( withCase, withoutCase );
                } )
                .FirstOrDefault();
 
            if ( declaration != null )
            {
                index = declaration.Index;
            }
        }
    }
    public override string OnCommit( IVsTextView textView, string textSoFar, char commitCharacter, int index, ref TextSpan initialExtent )
    {
        string committed = base.OnCommit( textView, textSoFar, commitCharacter, index, ref initialExtent );
        InsertUsedItem( committed );
        return committed;
    }
}