<p>Error: Incomplete information for this server ID. </p> Help:External Data - Edge of Darkness Wiki

Help:External Data

From Edge of Darkness Wiki

External Data
Jump to: navigation, search

Template:TNT

The External Data extension allows MediaWiki pages to retrieve, filter, and format structured data from one or more sources. These sources can include external URLs, regular wiki pages, uploaded files, files on the local server, databases or LDAP directories.

Contents

UNIQ34625eb8546c755f-h-0--QINUParser functions

The extension has nine parser functions:

UNIQ34625eb8546c755f-h-1--QINUDownload

You can download the External Data code, in .zip format, here.

You can also download the code directly via Git from the MediaWiki source code repository. From a command line, you can call the following:

<syntaxhighlight lang="bash">git clone https://gerrit.wikimedia.org/r/p/mediawiki/extensions/ExternalData.git</syntaxhighlight>

You can also view the code online here.

Template:SB Extension

UNIQ34625eb8546c755f-h-2--QINUInstallation

To install this extension, create an 'ExternalData' directory (either by extracting a compressed file or downloading via Git), and place this directory within the main MediaWiki 'extensions' directory. Then, in the file 'LocalSettings.php', add the following line:

<source lang="php">require_once "$IP/extensions/ExternalData/ExternalData.php";</source>


UNIQ34625eb8546c755f-h-3--QINURetrieving data

Data can be retrieved from three different sources: from a web page containing structured data (including a page on the wiki itself), from a database, and from an LDAP server.


UNIQ34625eb8546c755f-h-4--QINUData caching

You can configure External Data to cache the data contained in the URLs that it accesses, both to speed up retrieval of values and to reduce the load on the system whose data is being accessed. To do this, you can run the SQL contained in the extension file 'ExternalData.sql' in your database, which will create the table 'ed_url_cache', then add the following to your LocalSettings.php file, after the inclusion of External Data: <source lang="php">$edgCacheTable = 'ed_url_cache';</source>

You should also add a line like the following, to set the expiration time of the cache, in seconds; this example line will cache the data for a week: <source lang="php">$edgCacheExpireTime = 7 * 24 * 60 * 60;</source>


UNIQ34625eb8546c755f-h-5--QINU#get_db_data - retrieve data from a database

The parser function #get_db_data allows retrieval of data from external databases. This function executes a simple SELECT statement and assigns the results to local variables that can then be used with the #external_value or #for_external_table functions.

A note about security: - If you are going to use #get_db_data you should think about the security implications. Configuring a database in LocalSettings.php will allow anyone with edit access to your wiki to run SQL select statements against that database. You should use a database user that has the minimum permissions for what you are trying to achieve. It is possible that complex SQL constructions could be passed to this function to cause it to do things vastly different from what it was designed for.

UNIQ34625eb8546c755f-h-6--QINUConfiguration

Each database being accessed needs to be configured separately in LocalSettings.php. For normal databases (i.e., everything except for SQLite), add the following stanza for each database:

<syntaxhighlight lang="php"> $edgDBServer['ID'] = "server URL"; $edgDBServerType['ID'] = "DB type"; $edgDBName['ID'] = "DB name"; $edgDBUser['ID'] = "username"; $edgDBPass['ID'] = "password"; </syntaxhighlight>

Where:

An example of a set of values would be: <source lang="php"> $edgDBServer['employee-db'] = "127.0.0.1"; $edgDBServerType['employee-db'] = "mysql"; $edgDBName['employee-db'] = "employeesDatabase"; $edgDBUser['employee-db'] = "guest"; $edgDBPass['employee-db'] = "p@ssw0rd"; </source>

The following optional settings can also be added: <source lang="php"> $edgDBFlags['id'] = "MediaWiki DB flags"; $edgDBTablePrefix['id'] = "table prefix"; </source>

Example values for these variables are: <source lang="php"> $edgDBFlags['employee-db'] = DBO_NOBUFFER & DBO_TRX; $edgDBTablePrefix['employee-db'] = "emp_"; </source>

UNIQ34625eb8546c755f-h-7--QINUUsage

To get data from an external database, call the following:

<poem>

</poem>

An explanation of the fields:

An example call, using the "employee database" example from above: 

{{#get_db_data:
db=employee-db
|from=tblPersonalInfo
|where=employeeID='{{{id}}}'
|limit=50
|order by=employeeName ASC
|data=dbemail=employeeEmail,name=employeeName
}}


Displaying data

Once you have retrieved the data onto the page, from any source, there are two ways to display it on the page: #external_value and #for_external_table.

Displaying individual values

If this call retrieved a single value for each variable specified, you can call the following:

<poem></poem>

As an example, this page contains the following text: 

{{#get_web_data:url=http://discoursedb.org/GermanyCSV.txt
    |format=csv with header|data=bordered countries=Borders,population=Population,area=Area,capital=Capital}}

* Germany borders the following countries:
{{#arraymap:{{#external_value:bordered countries}}|,|x|[[x]]}}.
* Germany has population {{#external_value:population}}.
* Germany has area {{#external_value:area}}.
* Its capital is {{#external_value:capital}}.

The page gets data from this URL, which contains the following text: 

"357,050 km²","Austria,Belgium,Czech Republic,Denmark,France,Luxembourg,Netherlands,Poland,Switzerland",Berlin,"82,411,001"

The page then uses #external_value to display the 'bordered countries' and 'population' values; although it uses the #arraymap function, defined by the Semantic Forms extension, to apply some transformations to the 'bordered countries' value (you can ignore this detail if you want).

By default, #external_value displays an error message if it is called for a variable that has not been set, or if the specified data source is inaccessible, or the data source does not contain any data. You can disable the error message by adding the following to LocalSettings.php: <syntaxhighlight lang="php"> $edgExternalValueVerbose = false; </syntaxhighlight>

Displaying a table of values

The data returned by #get_web_data or #get_db_data (#get_ldap_data doesn't support this feature) can also be a "table" of data (many values per field), instead of just a single "row" (one value per field). In this case, you can display it using one of either the functions #for_external_table or #display_external_table.

#for_external_table

This URL contains information similar to that above, but for a few countries instead of just one. Calling #get_web_data with this URL, with the same format as above, will set the local variables to contain arrays of data, rather than single values. You can then call #for_external_table, which has the following format:

<poem></poem>

...where "expression" is a string that contains one or more variable names, surrounded by triple brackets. This string is then displayed for each retrieved "row" of data.

For an example, this page contains a call to #get_web_data for the URL mentioned above, followed by this call:

<poem>{| class="wikitable" ! Name ! Borders ! Population ! Area |}</poem>

The call to #for_external_table holds a single row of a table, in wiki-text; it's surrounded by wiki-text to create the top and bottom of the table. The presence of "{{!}}" is a standard MediaWiki trick to display pipes from within parser functions; to get it to work, you just have to create a page called "Template:!" that contains a single pipe. There are much easier calls to #for_external_table that can be made, if you just want to display a line of text per data "row", but an HTML table is the standard approach.

There's one other interesting feature of #for_external_table, which is that it lets you modify specific values. You can URL-encode values by calling them with {{{field-name.urlencode}}} instead of just {{{field-name}}}, and similarly you can HTML-encode values by calling them with {{{field-name.htmlencode}}}.

As an example of the former, if you wanted to show links to Google searches on a set of terms retrieved, you could call: 

{{#for_external_table: http://www.google.com/search?q={{{term.urlencode}}} }}

This is required because standard parser functions can't be used within #for_external_table - so the following, for example, will not work: 

{{#for_external_table: http://www.google.com/search?q={{urlencode:{{{term}}}}} }}

#display_external_table

#display_external_table is similar in concept to #for_external_table, but it passes the values in each row to a template, which handles the display. This function is called as:

<poem>{{#display_external_table: template=template name |data=set of parameters, separated by commas |delimiter=delimiter }}</poem>

An explanation of the parameters:

For example, to display the data from the previous example in a table as before, you could create a template called "Country info row", that had the parameters "Country name", "Countries bordered", "Population" and "Area", and then call the following: 

{| class="wikitable"
! Name
! Borders
! Population
! Area 
{{#display_external_table:template=Country info row|data=Country name=name,Countries bordered=borders,Population=population,Area=area}}
|}

The template should then contain wikitext like the following: 

|-
|{{{Country name}}}
|{{{Countries bordered}}}
|{{{Population}}}
|{{{Area}}}

Clearing data

You can also clear all external data that has already been retrieved, so that it doesn't conflict with calls to retrieve external data further down the page. The most likely case in which this is useful is when data is retrieved and displayed in a template that is called more than once on a page. To clear the data, just call "{{#clear_external_data:}}". Note that the ":" has to be there at the end of the call, or else MediaWiki will ignore the parser function.

There is no way to clear the values for only one field; #clear_external_data erases the entire set of data.

Storing data

You can also use External Data to store a table of data that has been retrieved; you can do this using the storage capabilities of either the Semantic MediaWiki or Cargo extensions. Once the data has been stored, it can then be queried, aggregated, displayed etc. on the wiki by that extension.

Semantic MediaWiki

If you store data with Semantic MediaWiki, you should note a common problem, which is that the data stored by SMW does not get automatically updated when the data coming from the external source changes. The best solution for this, assuming you expect the data to change over time, is to create a cron job to call the SMW maintenance script "SMW_refreshData.php" at regular intervals, such as once a day; that way, the data is never more than a day old.

To store a table of data using SMW, you can use the #store_external_table function. This function works as a hybrid of the #for_external_table function and one of either the #subobject function (defined in the Semantic MediaWiki extension) or the #set_internal function (defined in the Semantic Internal Objects extension). The function used depends on whether Semantic MediaWiki uses SQLStore2 or SQLStore3; if SQLStore2 is used, #set_internal is used, so Semantic Internal Objects must be installed. The syntax of #store_external_table is very similar to the syntax of #set_internal (see the Semantic Internal Objects page for documentation), but it loops over each row, and uses variables, in the same way as #for_external_table. You can see a demonstration of this function on the page Fruits semantic data; the call to #store_external_table on that page looks like: 

{{#store_external_table:Is fruit in
|Has name={{{name}}}
|Has color={{{color}}}
|Has shape={{{shape}}}
}}

Cargo

There is no special parser function for storing data via Cargo; instead you should simply use #display_external_table, and include Cargo storage code within the template called by that function. You can see an example of Cargo-based storage using #display_external_table here; it uses this template, and you can see the resulting data here.

Common problems

<source lang="php">$wgHTTPTimeout = 60;</source>

Version history

External Data is currently at version 1.8.2. See the entire version history.

Contributing

Bugs and feature requests

You can use either of these mailing lists for any questions, suggestions or bug reports about External Data:

With either mailing list, please add "[ED]" at the beginning of the subject line, to clarify the subject matter.

(Although the External Data extension in general is independent of Semantic MediaWiki, the fact that the two extensions are often used together means that External Data can be considered part of the Semantic MediaWiki "family" for communication purposes.)

You can also send specific code patches to Yaron Koren, at yaron57@gmail.com.

Translating

Translation of External Data is done through translatewiki.net. The translation for this extension can be found here. To add language values or change existing ones, you should create an account on translatewiki.net, then request permission from the administrators to translate a certain language or languages on this page (this is a very simple process). Once you have permission for a given language, you can log in and add or edit whatever messages you want to in that language.

See also

External links

Retrieved from "http://nwod.org/wiki/index.php/Help:External_Data"
Personal tools
Namespaces
Variants
Actions
Navigation
games
Toolbox