There are two relationships in a use case diagram – include and extend.

The extend relationship is used when one use case adds behavior to another use case. In this situation, the base case would be executed. Then, depending on what happens during execution of that one use case, one or more other use cases could possibly be executed. Note that it’s not required that an extension actually be executed – it’s just a possibility. This also indicates some level of reuse between the features.

The include relationship is used when one use case includes the functionality of another use case. The specific behavior is not important, but the end result is, which means that the included use case could have a different implementation, but the same steps and end result. In this type of relationship, the included use case is usually not available as a stand-alone use case.

Looking at your particular example, I developed two use case models that I feel are more appropriate. In both, I changed the actor from “Member” to “Librarian”. It appears that the person actually interacting with the software is not the library member, but the librarian.

In this example, the Librarian can perform two functions – check out a book or reserve a book. Both examples require the librarian to look up the book using the information provided by the member. However, the search functionality is not available as a stand-alone entity. I’m not sure that this is what you are going for.

In this example, the Librarian can perform three functions – looking up books, checking out books, and recording reservations. Both checking out books and recording reservations use the look up book use case, but add additional behaviors. The implementation details are left as an exercise for additional specification, but this indicates that the in order to either check out a book or place a reservation, the librarian must first look up the book. I believe this is more in line with the functionality you are describing in your question.

I happen to come across this lately while studying about OO design.

I have problem to understand the extension relation in the use case
diagrams, sometimes some relations seem extension.

There is an extension relationship if one use case extends the behavior of another. The former is the extension, and the latter is the base. In extend relationships, the base is independent and meaningful by itself. This is not the case for the extension–it depends on the base. It does not stand on its own.

In general what are rules of thumb to find out extension relation?

Use an extension relation when one use case has behavior that is optional, or when it has a behavior that executes only under some conditions.

Could the reservation be the extend of borrow?

If a member can make a reservation independently (for whatever reason) even if the book is available (like he reached his borrowing limit), then no, reservation cannot be the extend of borrow. Reserve can stand on its own. A member can make a reservation without having to borrow. Otherwise, yes, it is an extension of borrow because it executes only when borrowing a book that is unavailable. I hope this helps.

Oh, and in the case that reserve is an extension of borrow, don’t draw it like you did in your diagram. To show extend relationship, the extension connects to the base with an open arrow. Add the keyword extend. The arrow points from the extension to the base.

I say you shouldn’t bother trying to figure out if you should use extends/includes. That only causes you to lose the focus of what you are really trying to accomplish, which is describe what your system has to be able to do. They have their “rare” beneficial uses but that’s the exception. You certainly don’t need them for each trivial “common” step.

I used to mull over those details until I realized “in hindsight” that they don’t really add much value to the overall process and focusing on them ends up pushing you towards implementation details that you really shouldn’t be concerned with at the use-case level. For example, while it may be that your design will reserve a book the same way no matter how you got there, once you start design you very well may discover that it can’t be done the same way. Now, you’ve got to “un-extend” the use-case because you made a design decision at the requirements level.

When it is appropriate to use extends/includes YOU WILL KNOW, you won’t have to ask yourself should I use it or not.

To directly answer your question “what are the rules”: If it isn’t clear-cut then don’t bother.

Also, just as an FYI, steps 1 and 2 should simply be “Member requests to Reserve Book”/”Member requests to Borrow Book”. Your descriptions have no clear initiating action to start each particular use-case, which is essential for writing good and useful use-cases.

Plus there’s no need to mention “information of the book” that’s already implied by the request. Once you start analyzing the use-case and doing the design then you’ll identify the information that is required by the request.